2023-03-22 15:53:30,097 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-22 15:53:30,384 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36 2023-03-22 15:53:30,398 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-22 15:53:30,399 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/cluster_73db91c0-5f76-0b43-c651-cc63b18b2aa1, deleteOnExit=true 2023-03-22 15:53:30,399 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-22 15:53:30,399 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/test.cache.data in system properties and HBase conf 2023-03-22 15:53:30,400 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/hadoop.tmp.dir in system properties and HBase conf 2023-03-22 15:53:30,400 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/hadoop.log.dir in system properties and HBase conf 2023-03-22 15:53:30,400 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-22 15:53:30,401 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-22 15:53:30,401 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-22 15:53:30,524 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-22 15:53:30,941 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-22 15:53:30,944 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-22 15:53:30,945 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-22 15:53:30,945 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-22 15:53:30,945 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-22 15:53:30,946 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-22 15:53:30,946 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-22 15:53:30,946 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-22 15:53:30,946 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-22 15:53:30,947 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-22 15:53:30,947 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/nfs.dump.dir in system properties and HBase conf 2023-03-22 15:53:30,948 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/java.io.tmpdir in system properties and HBase conf 2023-03-22 15:53:30,948 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-22 15:53:30,949 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-22 15:53:30,949 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-22 15:53:31,554 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-22 15:53:31,558 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-22 15:53:31,847 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-22 15:53:32,029 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-22 15:53:32,050 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-22 15:53:32,090 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-22 15:53:32,124 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/java.io.tmpdir/Jetty_localhost_37767_hdfs____.h9m32a/webapp 2023-03-22 15:53:32,274 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37767 2023-03-22 15:53:32,328 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-22 15:53:32,329 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-22 15:53:32,808 WARN [Listener at localhost/33875] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-22 15:53:32,926 WARN [Listener at localhost/33875] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-22 15:53:32,946 WARN [Listener at localhost/33875] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-22 15:53:32,952 INFO [Listener at localhost/33875] log.Slf4jLog(67): jetty-6.1.26 2023-03-22 15:53:32,956 INFO [Listener at localhost/33875] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/java.io.tmpdir/Jetty_localhost_33007_datanode____2hqtdq/webapp 2023-03-22 15:53:33,055 INFO [Listener at localhost/33875] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33007 2023-03-22 15:53:33,430 WARN [Listener at localhost/43289] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-22 15:53:34,007 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe8b9c4e99b77d23b: Processing first storage report for DS-e3751bd7-c93c-4d1b-9567-6ce5468ada3f from datanode 4568db8c-3472-4b46-8130-137a8e5fe635 2023-03-22 15:53:34,009 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe8b9c4e99b77d23b: from storage DS-e3751bd7-c93c-4d1b-9567-6ce5468ada3f node DatanodeRegistration(127.0.0.1:40387, datanodeUuid=4568db8c-3472-4b46-8130-137a8e5fe635, infoPort=35011, infoSecurePort=0, ipcPort=43289, storageInfo=lv=-57;cid=testClusterID;nsid=1120029640;c=1679500411633), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-22 15:53:34,009 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe8b9c4e99b77d23b: Processing first storage report for DS-7e9ec590-a6e9-4e66-bc98-6b5755a6b858 from datanode 4568db8c-3472-4b46-8130-137a8e5fe635 2023-03-22 15:53:34,009 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe8b9c4e99b77d23b: from storage DS-7e9ec590-a6e9-4e66-bc98-6b5755a6b858 node DatanodeRegistration(127.0.0.1:40387, datanodeUuid=4568db8c-3472-4b46-8130-137a8e5fe635, infoPort=35011, infoSecurePort=0, ipcPort=43289, storageInfo=lv=-57;cid=testClusterID;nsid=1120029640;c=1679500411633), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-22 15:53:34,010 DEBUG [Listener at localhost/43289] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36 2023-03-22 15:53:34,094 INFO [Listener at localhost/43289] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/cluster_73db91c0-5f76-0b43-c651-cc63b18b2aa1/zookeeper_0, clientPort=58225, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/cluster_73db91c0-5f76-0b43-c651-cc63b18b2aa1/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/cluster_73db91c0-5f76-0b43-c651-cc63b18b2aa1/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-22 15:53:34,107 INFO [Listener at localhost/43289] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=58225 2023-03-22 15:53:34,115 INFO [Listener at localhost/43289] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-22 15:53:34,117 INFO [Listener at localhost/43289] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-22 15:53:34,805 INFO [Listener at localhost/43289] util.FSUtils(471): Created version file at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98 with version=8 2023-03-22 15:53:34,805 INFO [Listener at localhost/43289] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/hbase-staging 2023-03-22 15:53:35,112 INFO [Listener at localhost/43289] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-22 15:53:35,585 INFO [Listener at localhost/43289] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-22 15:53:35,623 INFO [Listener at localhost/43289] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-22 15:53:35,623 INFO [Listener at localhost/43289] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-22 15:53:35,624 INFO [Listener at localhost/43289] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-22 15:53:35,624 INFO [Listener at localhost/43289] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-22 15:53:35,624 INFO [Listener at localhost/43289] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-22 15:53:35,788 INFO [Listener at localhost/43289] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-22 15:53:35,878 DEBUG [Listener at localhost/43289] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-22 15:53:35,974 INFO [Listener at localhost/43289] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37827 2023-03-22 15:53:35,984 INFO [Listener at localhost/43289] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-22 15:53:35,986 INFO [Listener at localhost/43289] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-22 15:53:36,007 INFO [Listener at localhost/43289] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37827 connecting to ZooKeeper ensemble=127.0.0.1:58225 2023-03-22 15:53:36,049 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:378270x0, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-22 15:53:36,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37827-0x1010cee76450000 connected 2023-03-22 15:53:36,081 DEBUG [Listener at localhost/43289] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-22 15:53:36,082 DEBUG [Listener at localhost/43289] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-22 15:53:36,086 DEBUG [Listener at localhost/43289] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-22 15:53:36,096 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37827 2023-03-22 15:53:36,096 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37827 2023-03-22 15:53:36,096 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37827 2023-03-22 15:53:36,097 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37827 2023-03-22 15:53:36,097 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37827 2023-03-22 15:53:36,103 INFO [Listener at localhost/43289] master.HMaster(444): hbase.rootdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98, hbase.cluster.distributed=false 2023-03-22 15:53:36,186 INFO [Listener at localhost/43289] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-22 15:53:36,186 INFO [Listener at localhost/43289] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-22 15:53:36,186 INFO [Listener at localhost/43289] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-22 15:53:36,187 INFO [Listener at localhost/43289] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-22 15:53:36,187 INFO [Listener at localhost/43289] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-22 15:53:36,187 INFO [Listener at localhost/43289] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-22 15:53:36,194 INFO [Listener at localhost/43289] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-22 15:53:36,197 INFO [Listener at localhost/43289] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38637 2023-03-22 15:53:36,199 INFO [Listener at localhost/43289] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-22 15:53:36,205 DEBUG [Listener at localhost/43289] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-22 15:53:36,206 INFO [Listener at localhost/43289] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-22 15:53:36,209 INFO [Listener at localhost/43289] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-22 15:53:36,211 INFO [Listener at localhost/43289] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38637 connecting to ZooKeeper ensemble=127.0.0.1:58225 2023-03-22 15:53:36,214 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:386370x0, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-22 15:53:36,215 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:38637-0x1010cee76450001 connected 2023-03-22 15:53:36,215 DEBUG [Listener at localhost/43289] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-22 15:53:36,216 DEBUG [Listener at localhost/43289] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-22 15:53:36,217 DEBUG [Listener at localhost/43289] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-22 15:53:36,218 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38637 2023-03-22 15:53:36,219 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38637 2023-03-22 15:53:36,219 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38637 2023-03-22 15:53:36,219 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38637 2023-03-22 15:53:36,220 DEBUG [Listener at localhost/43289] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38637 2023-03-22 15:53:36,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:53:36,235 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-22 15:53:36,236 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:53:36,256 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-22 15:53:36,256 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-22 15:53:36,256 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-22 15:53:36,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-22 15:53:36,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37827,1679500414949 from backup master directory 2023-03-22 15:53:36,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-22 15:53:36,262 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:53:36,262 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-22 15:53:36,263 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-22 15:53:36,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:53:36,266 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-22 15:53:36,267 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-22 15:53:36,355 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/hbase.id with ID: 0bfc5956-e78c-4ecb-b687-d1bd3c89117f 2023-03-22 15:53:36,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-22 15:53:36,416 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-22 15:53:36,459 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x64378486 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:53:36,498 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@58a93e15, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:53:36,521 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-22 15:53:36,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-22 15:53:36,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-22 15:53:36,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-22 15:53:36,547 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-22 15:53:36,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-22 15:53:36,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-22 15:53:36,585 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/data/master/store-tmp 2023-03-22 15:53:37,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:37,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-22 15:53:37,016 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-22 15:53:37,016 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-22 15:53:37,016 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-22 15:53:37,016 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-22 15:53:37,016 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-22 15:53:37,016 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-22 15:53:37,018 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/WALs/jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:53:37,040 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37827%2C1679500414949, suffix=, logDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/WALs/jenkins-hbase4.apache.org,37827,1679500414949, archiveDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/oldWALs, maxLogs=10 2023-03-22 15:53:37,094 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40387,DS-e3751bd7-c93c-4d1b-9567-6ce5468ada3f,DISK] 2023-03-22 15:53:37,106 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-22 15:53:37,184 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/WALs/jenkins-hbase4.apache.org,37827,1679500414949/jenkins-hbase4.apache.org%2C37827%2C1679500414949.1679500417050 2023-03-22 15:53:37,184 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40387,DS-e3751bd7-c93c-4d1b-9567-6ce5468ada3f,DISK]] 2023-03-22 15:53:37,185 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:53:37,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:37,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-22 15:53:37,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-22 15:53:37,277 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-22 15:53:37,285 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-22 15:53:37,316 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-22 15:53:37,331 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:37,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-22 15:53:37,340 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-22 15:53:37,360 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-22 15:53:37,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:53:37,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66242290, jitterRate=-0.012912958860397339}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:53:37,373 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-22 15:53:37,375 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-22 15:53:37,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-22 15:53:37,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-22 15:53:37,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-22 15:53:37,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-22 15:53:37,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 35 msec 2023-03-22 15:53:37,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-22 15:53:37,466 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-22 15:53:37,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-22 15:53:37,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-22 15:53:37,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-22 15:53:37,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-22 15:53:37,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-22 15:53:37,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-22 15:53:37,518 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-22 15:53:37,519 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-22 15:53:37,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-22 15:53:37,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-22 15:53:37,536 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-22 15:53:37,536 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-22 15:53:37,536 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-22 15:53:37,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,37827,1679500414949, sessionid=0x1010cee76450000, setting cluster-up flag (Was=false) 2023-03-22 15:53:37,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-22 15:53:37,558 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-22 15:53:37,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:53:37,565 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-22 15:53:37,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-22 15:53:37,572 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:53:37,575 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/.tmp 2023-03-22 15:53:37,626 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(952): ClusterId : 0bfc5956-e78c-4ecb-b687-d1bd3c89117f 2023-03-22 15:53:37,631 DEBUG [RS:0;jenkins-hbase4:38637] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-22 15:53:37,637 DEBUG [RS:0;jenkins-hbase4:38637] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-22 15:53:37,637 DEBUG [RS:0;jenkins-hbase4:38637] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-22 15:53:37,642 DEBUG [RS:0;jenkins-hbase4:38637] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-22 15:53:37,643 DEBUG [RS:0;jenkins-hbase4:38637] zookeeper.ReadOnlyZKClient(139): Connect 0x4cb57066 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:53:37,649 DEBUG [RS:0;jenkins-hbase4:38637] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@42728780, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:53:37,650 DEBUG [RS:0;jenkins-hbase4:38637] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@464d1b16, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-22 15:53:37,676 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:38637 2023-03-22 15:53:37,681 INFO [RS:0;jenkins-hbase4:38637] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-22 15:53:37,681 INFO [RS:0;jenkins-hbase4:38637] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-22 15:53:37,681 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1023): About to register with Master. 2023-03-22 15:53:37,685 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37827,1679500414949 with isa=jenkins-hbase4.apache.org/172.31.14.131:38637, startcode=1679500416185 2023-03-22 15:53:37,702 DEBUG [RS:0;jenkins-hbase4:38637] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-22 15:53:37,714 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-22 15:53:37,727 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-22 15:53:37,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-22 15:53:37,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-22 15:53:37,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-22 15:53:37,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-22 15:53:37,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-22 15:53:37,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,732 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1679500447732 2023-03-22 15:53:37,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-22 15:53:37,738 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-22 15:53:37,738 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-22 15:53:37,744 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-22 15:53:37,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-22 15:53:37,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-22 15:53:37,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-22 15:53:37,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-22 15:53:37,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-22 15:53:37,755 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:37,757 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-22 15:53:37,758 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46095, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-22 15:53:37,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-22 15:53:37,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-22 15:53:37,762 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-22 15:53:37,762 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-22 15:53:37,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1679500417764,5,FailOnTimeoutGroup] 2023-03-22 15:53:37,764 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1679500417764,5,FailOnTimeoutGroup] 2023-03-22 15:53:37,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:37,764 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-22 15:53:37,766 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:37,766 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:37,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-03-22 15:53:37,800 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(2829): Master is not running yet 2023-03-22 15:53:37,801 WARN [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-03-22 15:53:37,902 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37827,1679500414949 with isa=jenkins-hbase4.apache.org/172.31.14.131:38637, startcode=1679500416185 2023-03-22 15:53:37,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:37,914 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98 2023-03-22 15:53:37,915 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:33875 2023-03-22 15:53:37,915 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-22 15:53:37,922 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-22 15:53:37,923 DEBUG [RS:0;jenkins-hbase4:38637] zookeeper.ZKUtil(162): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:37,923 WARN [RS:0;jenkins-hbase4:38637] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-22 15:53:37,923 INFO [RS:0;jenkins-hbase4:38637] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-22 15:53:37,924 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1947): logDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/WALs/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:37,925 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38637,1679500416185] 2023-03-22 15:53:37,934 DEBUG [RS:0;jenkins-hbase4:38637] zookeeper.ZKUtil(162): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:37,947 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-22 15:53:37,958 INFO [RS:0;jenkins-hbase4:38637] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-22 15:53:37,977 INFO [RS:0;jenkins-hbase4:38637] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-22 15:53:37,981 INFO [RS:0;jenkins-hbase4:38637] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:53:37,981 INFO [RS:0;jenkins-hbase4:38637] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:37,982 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-22 15:53:37,989 INFO [RS:0;jenkins-hbase4:38637] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-22 15:53:37,989 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,990 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,990 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,990 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,990 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,990 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-22 15:53:37,990 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,990 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,991 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,991 DEBUG [RS:0;jenkins-hbase4:38637] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-22 15:53:37,991 INFO [RS:0;jenkins-hbase4:38637] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:37,992 INFO [RS:0;jenkins-hbase4:38637] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:37,992 INFO [RS:0;jenkins-hbase4:38637] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:38,008 INFO [RS:0;jenkins-hbase4:38637] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-22 15:53:38,011 INFO [RS:0;jenkins-hbase4:38637] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38637,1679500416185-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:38,027 INFO [RS:0;jenkins-hbase4:38637] regionserver.Replication(203): jenkins-hbase4.apache.org,38637,1679500416185 started 2023-03-22 15:53:38,028 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,38637,1679500416185, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38637, sessionid=0x1010cee76450001 2023-03-22 15:53:38,028 DEBUG [RS:0;jenkins-hbase4:38637] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-22 15:53:38,028 DEBUG [RS:0;jenkins-hbase4:38637] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:38,028 DEBUG [RS:0;jenkins-hbase4:38637] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38637,1679500416185' 2023-03-22 15:53:38,028 DEBUG [RS:0;jenkins-hbase4:38637] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:38,029 DEBUG [RS:0;jenkins-hbase4:38637] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:38,029 DEBUG [RS:0;jenkins-hbase4:38637] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-22 15:53:38,030 DEBUG [RS:0;jenkins-hbase4:38637] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-22 15:53:38,030 DEBUG [RS:0;jenkins-hbase4:38637] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:38,030 DEBUG [RS:0;jenkins-hbase4:38637] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38637,1679500416185' 2023-03-22 15:53:38,030 DEBUG [RS:0;jenkins-hbase4:38637] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-22 15:53:38,030 DEBUG [RS:0;jenkins-hbase4:38637] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-22 15:53:38,031 DEBUG [RS:0;jenkins-hbase4:38637] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-22 15:53:38,031 INFO [RS:0;jenkins-hbase4:38637] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-22 15:53:38,031 INFO [RS:0;jenkins-hbase4:38637] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-22 15:53:38,142 INFO [RS:0;jenkins-hbase4:38637] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38637%2C1679500416185, suffix=, logDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/WALs/jenkins-hbase4.apache.org,38637,1679500416185, archiveDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/oldWALs, maxLogs=32 2023-03-22 15:53:38,162 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40387,DS-e3751bd7-c93c-4d1b-9567-6ce5468ada3f,DISK] 2023-03-22 15:53:38,166 INFO [RS:0;jenkins-hbase4:38637] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/WALs/jenkins-hbase4.apache.org,38637,1679500416185/jenkins-hbase4.apache.org%2C38637%2C1679500416185.1679500418145 2023-03-22 15:53:38,166 DEBUG [RS:0;jenkins-hbase4:38637] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40387,DS-e3751bd7-c93c-4d1b-9567-6ce5468ada3f,DISK]] 2023-03-22 15:53:38,186 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-22 15:53:38,187 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-22 15:53:38,188 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98 2023-03-22 15:53:38,601 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:38,604 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-22 15:53:38,606 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/info 2023-03-22 15:53:38,607 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-22 15:53:38,609 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:38,609 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-22 15:53:38,612 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/rep_barrier 2023-03-22 15:53:38,612 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-22 15:53:38,613 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:38,614 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-22 15:53:38,616 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/table 2023-03-22 15:53:38,617 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-22 15:53:38,617 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:38,619 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740 2023-03-22 15:53:38,620 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740 2023-03-22 15:53:38,624 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-22 15:53:38,626 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-22 15:53:38,630 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:53:38,631 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74862221, jitterRate=0.11553402245044708}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-22 15:53:38,631 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-22 15:53:38,631 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-22 15:53:38,631 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-22 15:53:38,631 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-22 15:53:38,631 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-22 15:53:38,631 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-22 15:53:38,632 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-22 15:53:38,633 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-22 15:53:38,638 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-22 15:53:38,638 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-22 15:53:38,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-22 15:53:38,659 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-22 15:53:38,661 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-22 15:53:38,813 DEBUG [jenkins-hbase4:37827] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-22 15:53:38,816 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38637,1679500416185, state=OPENING 2023-03-22 15:53:38,823 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-22 15:53:38,824 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-22 15:53:38,825 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-22 15:53:38,827 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:53:39,012 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:39,015 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-22 15:53:39,018 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54750, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-22 15:53:39,030 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-22 15:53:39,030 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-22 15:53:39,033 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38637%2C1679500416185.meta, suffix=.meta, logDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/WALs/jenkins-hbase4.apache.org,38637,1679500416185, archiveDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/oldWALs, maxLogs=32 2023-03-22 15:53:39,050 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40387,DS-e3751bd7-c93c-4d1b-9567-6ce5468ada3f,DISK] 2023-03-22 15:53:39,054 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/WALs/jenkins-hbase4.apache.org,38637,1679500416185/jenkins-hbase4.apache.org%2C38637%2C1679500416185.meta.1679500419035.meta 2023-03-22 15:53:39,054 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40387,DS-e3751bd7-c93c-4d1b-9567-6ce5468ada3f,DISK]] 2023-03-22 15:53:39,054 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:53:39,056 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-22 15:53:39,072 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-22 15:53:39,077 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-22 15:53:39,082 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-22 15:53:39,082 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:39,082 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-22 15:53:39,082 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-22 15:53:39,085 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-22 15:53:39,087 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/info 2023-03-22 15:53:39,087 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/info 2023-03-22 15:53:39,088 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-22 15:53:39,089 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:39,089 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-22 15:53:39,091 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/rep_barrier 2023-03-22 15:53:39,091 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/rep_barrier 2023-03-22 15:53:39,091 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-22 15:53:39,092 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:39,092 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-22 15:53:39,094 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/table 2023-03-22 15:53:39,094 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/table 2023-03-22 15:53:39,094 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-22 15:53:39,095 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:39,097 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740 2023-03-22 15:53:39,099 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740 2023-03-22 15:53:39,104 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-22 15:53:39,106 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-22 15:53:39,108 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59461948, jitterRate=-0.11394792795181274}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-22 15:53:39,108 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-22 15:53:39,121 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1679500419004 2023-03-22 15:53:39,141 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-22 15:53:39,142 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-22 15:53:39,143 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,38637,1679500416185, state=OPEN 2023-03-22 15:53:39,147 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-22 15:53:39,147 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-22 15:53:39,153 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-22 15:53:39,153 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,38637,1679500416185 in 320 msec 2023-03-22 15:53:39,159 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-22 15:53:39,159 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 508 msec 2023-03-22 15:53:39,165 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5320 sec 2023-03-22 15:53:39,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1679500419166, completionTime=-1 2023-03-22 15:53:39,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-22 15:53:39,167 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-03-22 15:53:39,238 DEBUG [hconnection-0x8ab6f4c-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:53:39,241 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54758, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:53:39,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-03-22 15:53:39,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1679500479258 2023-03-22 15:53:39,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1679500539258 2023-03-22 15:53:39,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 91 msec 2023-03-22 15:53:39,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37827,1679500414949-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:39,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37827,1679500414949-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:39,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37827,1679500414949-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:39,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37827, period=300000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:39,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-22 15:53:39,288 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-22 15:53:39,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-22 15:53:39,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-22 15:53:39,307 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-22 15:53:39,310 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:53:39,313 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:53:39,338 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:53:39,341 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c empty. 2023-03-22 15:53:39,342 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:53:39,342 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-22 15:53:39,767 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-22 15:53:39,769 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => ffeec34469c62843ee9cb5963621aa5c, NAME => 'hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:53:39,781 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:39,782 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing ffeec34469c62843ee9cb5963621aa5c, disabling compactions & flushes 2023-03-22 15:53:39,782 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:53:39,782 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:53:39,782 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. after waiting 0 ms 2023-03-22 15:53:39,782 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:53:39,782 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:53:39,782 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for ffeec34469c62843ee9cb5963621aa5c: 2023-03-22 15:53:39,786 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:53:39,800 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679500419789"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500419789"}]},"ts":"1679500419789"} 2023-03-22 15:53:39,827 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-22 15:53:39,828 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:53:39,833 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500419829"}]},"ts":"1679500419829"} 2023-03-22 15:53:39,838 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-22 15:53:39,850 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ffeec34469c62843ee9cb5963621aa5c, ASSIGN}] 2023-03-22 15:53:39,853 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ffeec34469c62843ee9cb5963621aa5c, ASSIGN 2023-03-22 15:53:39,855 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ffeec34469c62843ee9cb5963621aa5c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:53:40,006 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ffeec34469c62843ee9cb5963621aa5c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:40,006 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679500420006"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500420006"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500420006"}]},"ts":"1679500420006"} 2023-03-22 15:53:40,012 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure ffeec34469c62843ee9cb5963621aa5c, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:53:40,172 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:53:40,173 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ffeec34469c62843ee9cb5963621aa5c, NAME => 'hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:53:40,173 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:53:40,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:40,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:53:40,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:53:40,176 INFO [StoreOpener-ffeec34469c62843ee9cb5963621aa5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:53:40,179 DEBUG [StoreOpener-ffeec34469c62843ee9cb5963621aa5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c/info 2023-03-22 15:53:40,179 DEBUG [StoreOpener-ffeec34469c62843ee9cb5963621aa5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c/info 2023-03-22 15:53:40,179 INFO [StoreOpener-ffeec34469c62843ee9cb5963621aa5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ffeec34469c62843ee9cb5963621aa5c columnFamilyName info 2023-03-22 15:53:40,181 INFO [StoreOpener-ffeec34469c62843ee9cb5963621aa5c-1] regionserver.HStore(310): Store=ffeec34469c62843ee9cb5963621aa5c/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:40,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:53:40,184 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:53:40,189 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:53:40,193 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:53:40,194 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ffeec34469c62843ee9cb5963621aa5c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72039528, jitterRate=0.07347261905670166}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:53:40,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ffeec34469c62843ee9cb5963621aa5c: 2023-03-22 15:53:40,197 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c., pid=6, masterSystemTime=1679500420166 2023-03-22 15:53:40,201 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:53:40,201 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:53:40,203 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ffeec34469c62843ee9cb5963621aa5c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:40,204 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679500420202"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500420202"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500420202"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500420202"}]},"ts":"1679500420202"} 2023-03-22 15:53:40,211 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-22 15:53:40,211 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure ffeec34469c62843ee9cb5963621aa5c, server=jenkins-hbase4.apache.org,38637,1679500416185 in 196 msec 2023-03-22 15:53:40,215 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-22 15:53:40,215 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ffeec34469c62843ee9cb5963621aa5c, ASSIGN in 361 msec 2023-03-22 15:53:40,216 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:53:40,217 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500420217"}]},"ts":"1679500420217"} 2023-03-22 15:53:40,220 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-22 15:53:40,224 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:53:40,227 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 925 msec 2023-03-22 15:53:40,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-22 15:53:40,313 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-22 15:53:40,313 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-22 15:53:40,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-22 15:53:40,371 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-22 15:53:40,377 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 34 msec 2023-03-22 15:53:40,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-22 15:53:40,397 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-22 15:53:40,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-03-22 15:53:40,412 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-22 15:53:40,416 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-22 15:53:40,416 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.153sec 2023-03-22 15:53:40,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-22 15:53:40,420 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-22 15:53:40,420 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-22 15:53:40,420 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37827,1679500414949-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-22 15:53:40,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37827,1679500414949-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-22 15:53:40,432 DEBUG [Listener at localhost/43289] zookeeper.ReadOnlyZKClient(139): Connect 0x561bf69b to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:53:40,432 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-22 15:53:40,436 DEBUG [Listener at localhost/43289] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18f0808a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:53:40,450 DEBUG [hconnection-0xcc760f5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:53:40,461 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54762, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:53:40,474 INFO [Listener at localhost/43289] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:53:40,475 DEBUG [Listener at localhost/43289] zookeeper.ReadOnlyZKClient(139): Connect 0x6b586bb5 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:53:40,480 DEBUG [Listener at localhost/43289] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69d78ff9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:53:40,480 INFO [Listener at localhost/43289] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-22 15:53:40,502 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=209, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=295, ProcessCount=172, AvailableMemoryMB=7434 2023-03-22 15:53:40,513 DEBUG [Listener at localhost/43289] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:53:40,516 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48646, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:53:40,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-22 15:53:40,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:40,535 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:53:40,537 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:53:40,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-22 15:53:40,547 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:40,548 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d empty. 2023-03-22 15:53:40,550 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:40,551 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-22 15:53:40,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-22 15:53:40,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-22 15:53:40,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-22 15:53:40,975 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-22 15:53:40,981 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7aa87cf5a808cbf44085918270c1463d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:53:41,002 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:41,002 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 7aa87cf5a808cbf44085918270c1463d, disabling compactions & flushes 2023-03-22 15:53:41,002 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:41,002 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:41,002 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. after waiting 0 ms 2023-03-22 15:53:41,002 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:41,002 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:41,003 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:41,011 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:53:41,014 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679500421014"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500421014"}]},"ts":"1679500421014"} 2023-03-22 15:53:41,019 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-22 15:53:41,021 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:53:41,022 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500421021"}]},"ts":"1679500421021"} 2023-03-22 15:53:41,031 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-22 15:53:41,041 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=7aa87cf5a808cbf44085918270c1463d, ASSIGN}] 2023-03-22 15:53:41,046 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=7aa87cf5a808cbf44085918270c1463d, ASSIGN 2023-03-22 15:53:41,049 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=7aa87cf5a808cbf44085918270c1463d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:53:41,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-22 15:53:41,201 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=7aa87cf5a808cbf44085918270c1463d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:41,201 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679500421200"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500421200"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500421200"}]},"ts":"1679500421200"} 2023-03-22 15:53:41,206 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 7aa87cf5a808cbf44085918270c1463d, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:53:41,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:41,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7aa87cf5a808cbf44085918270c1463d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:53:41,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:41,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:41,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:41,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:41,402 INFO [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:41,405 DEBUG [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:41,405 DEBUG [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:41,432 INFO [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7aa87cf5a808cbf44085918270c1463d columnFamilyName family1 2023-03-22 15:53:41,437 INFO [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] regionserver.HStore(310): Store=7aa87cf5a808cbf44085918270c1463d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:41,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:41,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:41,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:41,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:53:41,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7aa87cf5a808cbf44085918270c1463d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67660779, jitterRate=0.00822417438030243}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:53:41,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:41,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d., pid=11, masterSystemTime=1679500421367 2023-03-22 15:53:41,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:41,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:41,465 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=7aa87cf5a808cbf44085918270c1463d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:41,465 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679500421465"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500421465"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500421465"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500421465"}]},"ts":"1679500421465"} 2023-03-22 15:53:41,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-22 15:53:41,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 7aa87cf5a808cbf44085918270c1463d, server=jenkins-hbase4.apache.org,38637,1679500416185 in 263 msec 2023-03-22 15:53:41,479 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-22 15:53:41,480 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=7aa87cf5a808cbf44085918270c1463d, ASSIGN in 435 msec 2023-03-22 15:53:41,481 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:53:41,482 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500421481"}]},"ts":"1679500421481"} 2023-03-22 15:53:41,484 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-22 15:53:41,490 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:53:41,493 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 960 msec 2023-03-22 15:53:41,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-22 15:53:41,668 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-22 15:53:41,700 DEBUG [hconnection-0x3666aa34-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:53:41,718 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54764, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:53:41,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:41,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:41,988 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:41,988 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:41,989 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:41,989 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:41,990 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:41,990 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:41,991 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:41,991 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:41,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:41,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:41,992 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:41,992 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:41,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:41,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:41,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:41,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:41,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:41,996 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:41,996 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:41,996 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:41,998 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:42,000 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:42,000 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:42,001 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:42,002 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:42,003 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-22 15:53:42,257 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511300b364bc4d3e04718b0b7b18cf5ffcfc7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511300b364bc4d3e04718b0b7b18cf5ffcfc7 2023-03-22 15:53:42,283 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/4486805946524859b3e050e19e3798b2 2023-03-22 15:53:42,317 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/4486805946524859b3e050e19e3798b2 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4486805946524859b3e050e19e3798b2 2023-03-22 15:53:42,329 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4486805946524859b3e050e19e3798b2, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-22 15:53:42,333 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 329ms, sequenceid=5, compaction requested=false 2023-03-22 15:53:42,335 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:42,339 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:42,339 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:42,339 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:42,340 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:42,340 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,340 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:42,340 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:42,343 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,343 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:42,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:42,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:42,345 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,345 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:42,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:42,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:42,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:42,347 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:42,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@245c9a51[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:42,347 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,349 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,349 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:42,349 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:42,349 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:42,350 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,350 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:42,355 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,355 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:42,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:42,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:42,355 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:42,356 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:42,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:42,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:42,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:42,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:42,363 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:42,363 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:42,363 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:42,363 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:42,363 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,363 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:42,366 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,366 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,367 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:42,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:42,368 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:42,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,370 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:42,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,379 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,379 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:42,379 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:42,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:42,379 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:42,380 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:42,380 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,380 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:42,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:42,380 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:42,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:42,381 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,381 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,383 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:42,383 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:42,483 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:42,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:42,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:42,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:42,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,528 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:42,528 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:42,530 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:42,530 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:42,530 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,531 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,532 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:42,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:42,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:42,532 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,533 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:42,533 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:42,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:42,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,534 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,535 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-22 15:53:42,535 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:42,535 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:42,535 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:42,535 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:42,536 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:42,536 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:42,536 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:42,536 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:42,536 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-22 15:53:42,973 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112846552b3c0a9f433d9c65a96e6306e720 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112846552b3c0a9f433d9c65a96e6306e720 2023-03-22 15:53:43,406 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/3d2c91c679504914b199d14a17bb8a68 2023-03-22 15:53:43,425 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/3d2c91c679504914b199d14a17bb8a68 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/3d2c91c679504914b199d14a17bb8a68 2023-03-22 15:53:43,444 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/3d2c91c679504914b199d14a17bb8a68, entries=1, sequenceid=9, filesize=5.0 K 2023-03-22 15:53:43,448 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 912ms, sequenceid=9, compaction requested=false 2023-03-22 15:53:43,448 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:43,448 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:43,448 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:43,450 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:43,450 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,450 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:43,450 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:43,454 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:43,454 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:43,455 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,455 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:43,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:43,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:43,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:43,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f51361e[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:43,458 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:43,458 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,461 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,461 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:43,461 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:43,461 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:43,461 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,461 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:43,463 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,463 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:43,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:43,464 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:43,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:43,464 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:43,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:43,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:43,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:43,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:43,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:43,475 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:43,475 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:43,475 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:43,475 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,475 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:43,481 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,481 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,481 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:43,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:43,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:43,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:43,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:43,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,490 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:43,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:43,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,493 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:43,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,570 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:43,570 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:43,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:43,571 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,571 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,571 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,571 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,571 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,571 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:43,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:43,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:43,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:43,571 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,572 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:43,572 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:43,672 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:43,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:43,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:43,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:43,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,690 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:43,690 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:43,690 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:43,690 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:43,691 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,691 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,693 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:43,693 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:43,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,695 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:43,695 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:43,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:43,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:43,702 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:43,702 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:43,703 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:43,703 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:43,703 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:43,703 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:43,703 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:43,703 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:43,954 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-22 15:53:43,955 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-22 15:53:44,173 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201fe445a5829c84cb3af67d69be94197b9 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201fe445a5829c84cb3af67d69be94197b9 2023-03-22 15:53:44,198 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/0e65ce90ee3c4a21bb7232b2d950fe2b 2023-03-22 15:53:44,213 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/0e65ce90ee3c4a21bb7232b2d950fe2b as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/0e65ce90ee3c4a21bb7232b2d950fe2b 2023-03-22 15:53:44,224 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/0e65ce90ee3c4a21bb7232b2d950fe2b, entries=1, sequenceid=13, filesize=5.0 K 2023-03-22 15:53:44,226 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 523ms, sequenceid=13, compaction requested=true 2023-03-22 15:53:44,226 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:44,226 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:44,226 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:44,227 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:44,227 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,227 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:44,227 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:44,230 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,230 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:44,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:44,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:44,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:44,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:44,235 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:44,235 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bb88206[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:44,235 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:44,235 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,237 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,238 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:44,238 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:44,238 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:44,239 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,239 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:44,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:44,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:44,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:44,241 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:44,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:44,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:44,247 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:44,247 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:44,247 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:44,247 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,247 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:44,249 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,249 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:44,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:44,249 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:44,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:44,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:44,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:44,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,251 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:44,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,253 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:44,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,259 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:44,259 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,259 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:44,259 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:44,259 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:44,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:44,259 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:44,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:44,259 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,260 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,260 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,260 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,260 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:44,261 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:44,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:44,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:44,361 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:44,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:44,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:44,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:44,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,384 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:44,384 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:44,385 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:44,385 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:44,386 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,386 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,393 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:44,393 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:44,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:44,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,394 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,394 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:44,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:44,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,400 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-22 15:53:44,400 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,400 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:44,400 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:44,401 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:44,401 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:44,401 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:44,401 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:44,401 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:44,401 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:44,642 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:53:44,885 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512059c7c5673531e470093605fdf5909b07e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512059c7c5673531e470093605fdf5909b07e 2023-03-22 15:53:44,925 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/4c6f0a007a854c8a9d0991ca2f6120fd 2023-03-22 15:53:44,935 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/4c6f0a007a854c8a9d0991ca2f6120fd as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4c6f0a007a854c8a9d0991ca2f6120fd 2023-03-22 15:53:44,951 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4c6f0a007a854c8a9d0991ca2f6120fd, entries=1, sequenceid=17, filesize=5.0 K 2023-03-22 15:53:44,953 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 552ms, sequenceid=17, compaction requested=true 2023-03-22 15:53:44,954 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:44,954 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:44,954 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:44,954 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:44,954 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,954 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:44,955 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:44,958 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,958 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:44,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:44,959 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,959 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:44,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:44,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:44,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:44,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a942e99[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:44,973 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:44,973 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,975 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,976 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:44,976 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:44,976 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:44,976 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,976 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:44,981 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:44,981 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:44,981 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:44,982 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:44,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:44,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:44,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:44,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:44,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:44,987 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:44,987 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:44,987 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:44,987 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,987 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:44,990 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,990 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:44,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:44,990 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:44,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:44,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:44,991 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:44,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:44,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,994 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:44,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:44,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:44,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,004 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:45,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:45,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:45,004 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:45,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:45,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:45,016 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,016 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,016 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,017 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,017 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,017 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,017 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,017 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:45,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:45,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:45,020 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-22 15:53:45,021 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:45,021 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:45,106 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-22 15:53:45,107 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-22 15:53:45,121 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:45,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:45,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:45,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:45,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,137 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:45,137 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:45,138 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:45,138 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:45,139 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,139 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,142 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,142 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:45,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:45,142 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,142 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:45,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-22 15:53:45,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,145 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:45,145 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:45,145 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:45,146 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:45,146 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:45,146 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:45,146 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:45,603 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283db2d435a45041b88d0cb8be22ed7c37 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283db2d435a45041b88d0cb8be22ed7c37 2023-03-22 15:53:45,622 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/028506fc516f445ab7cd8f58453264e0 2023-03-22 15:53:45,637 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/028506fc516f445ab7cd8f58453264e0 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/028506fc516f445ab7cd8f58453264e0 2023-03-22 15:53:45,647 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/028506fc516f445ab7cd8f58453264e0, entries=1, sequenceid=21, filesize=5.0 K 2023-03-22 15:53:45,649 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 503ms, sequenceid=21, compaction requested=true 2023-03-22 15:53:45,649 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:45,649 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:45,649 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:45,651 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:45,651 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,651 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:45,651 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:45,657 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:45,658 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:45,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:45,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:45,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1adaa988[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:45,661 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:45,662 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,663 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,663 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:45,663 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,664 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:45,664 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:45,664 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:45,666 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,666 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:45,666 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:45,667 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:45,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:45,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:45,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:45,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:45,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:45,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:45,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:45,678 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:45,678 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:45,678 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:45,678 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,678 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:45,680 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,680 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:45,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:45,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,680 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:45,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:45,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:45,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:45,681 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:45,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,686 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:45,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,691 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,691 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:45,691 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,691 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,691 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:45,691 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:45,692 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,692 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,692 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:45,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:45,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:45,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:45,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:45,692 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:45,693 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:45,694 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:45,794 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:45,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:45,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:45,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:45,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,814 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:45,814 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:45,815 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:45,815 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:45,816 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,816 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,817 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:45,818 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:45,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:45,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:45,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,820 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,820 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:45,822 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-22 15:53:45,822 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,822 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:45,822 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:45,823 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:45,826 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:45,826 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:45,826 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:45,826 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:45,827 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:45,896 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231432cc91b70614694bcde226575077c5b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231432cc91b70614694bcde226575077c5b 2023-03-22 15:53:45,918 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/084f5fa86c9d43dea09a65c36df4b8f7 2023-03-22 15:53:45,930 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/084f5fa86c9d43dea09a65c36df4b8f7 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/084f5fa86c9d43dea09a65c36df4b8f7 2023-03-22 15:53:45,942 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/084f5fa86c9d43dea09a65c36df4b8f7, entries=1, sequenceid=25, filesize=5.0 K 2023-03-22 15:53:45,945 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 118ms, sequenceid=25, compaction requested=true 2023-03-22 15:53:45,945 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:45,945 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:45,945 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:45,946 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:45,946 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,946 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:45,946 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:45,948 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,948 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:45,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:45,949 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,949 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:45,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:45,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:45,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:45,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:45,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ccd8d0b[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:45,952 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:45,952 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,956 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,956 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:45,956 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:45,956 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:45,957 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,957 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:45,960 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:45,960 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,960 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:45,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:45,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:45,960 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:45,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:45,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:45,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:45,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:45,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:45,975 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:45,975 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:45,975 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:45,975 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,975 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:45,977 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,977 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:45,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,978 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:45,978 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:45,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:45,980 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,983 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:45,983 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:45,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:45,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:45,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:45,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:45,983 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,984 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,984 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,984 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:45,984 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,984 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,984 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:45,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:45,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:45,984 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:45,984 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:45,984 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:45,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-22 15:53:46,085 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:46,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:46,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:46,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:46,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,100 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:46,100 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:46,102 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:46,102 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:46,102 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,103 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:46,104 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:46,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:46,106 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,106 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:46,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:46,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,108 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,109 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-22 15:53:46,109 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,109 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:46,109 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:46,110 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:46,112 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:46,112 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:46,112 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:46,112 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:46,112 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:46,567 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ae1ea3c0b00a4fa9bc617f20e7cbb066 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101ae1ea3c0b00a4fa9bc617f20e7cbb066 2023-03-22 15:53:46,585 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/c60fb9db965d40c69eb69825fba1e198 2023-03-22 15:53:46,595 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/c60fb9db965d40c69eb69825fba1e198 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/c60fb9db965d40c69eb69825fba1e198 2023-03-22 15:53:46,608 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/c60fb9db965d40c69eb69825fba1e198, entries=1, sequenceid=29, filesize=5.0 K 2023-03-22 15:53:46,611 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 499ms, sequenceid=29, compaction requested=true 2023-03-22 15:53:46,611 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:46,611 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:46,611 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:46,612 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:46,612 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,612 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:46,612 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:46,615 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:46,616 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:46,617 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,617 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:46,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:46,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:46,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@434a8a26[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:46,619 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:46,619 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,621 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,621 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:46,621 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:46,621 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:46,621 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,621 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:46,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:46,623 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:46,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:46,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:46,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:46,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:46,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:46,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:46,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:46,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:46,632 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:46,632 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:46,632 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:46,632 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,632 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:46,634 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,635 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:46,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,635 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:46,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,636 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:46,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:46,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,639 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:46,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,647 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,648 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,648 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,648 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:46,648 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,648 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:46,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-22 15:53:46,648 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:46,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:46,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:46,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:46,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:46,648 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,649 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,649 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:46,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:46,649 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:46,650 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:46,750 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:46,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:46,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:46,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:46,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,763 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:46,764 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:46,764 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:46,764 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:46,765 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,765 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,766 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,766 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:46,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:46,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:46,767 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:46,767 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:46,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,769 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,769 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-22 15:53:46,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:46,771 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:46,771 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:46,772 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:46,774 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:46,774 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:46,774 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:46,774 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:46,774 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:47,232 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c7d1f72810b74140ba7ce7fe547a7392 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103c7d1f72810b74140ba7ce7fe547a7392 2023-03-22 15:53:47,275 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/5bcc23962f43430f9264b93b474ac7d9 2023-03-22 15:53:47,289 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/5bcc23962f43430f9264b93b474ac7d9 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/5bcc23962f43430f9264b93b474ac7d9 2023-03-22 15:53:47,303 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/5bcc23962f43430f9264b93b474ac7d9, entries=1, sequenceid=33, filesize=5.0 K 2023-03-22 15:53:47,304 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 530ms, sequenceid=33, compaction requested=true 2023-03-22 15:53:47,304 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:47,304 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:47,305 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:47,305 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:47,305 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,305 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:47,305 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:47,307 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,308 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:47,308 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,308 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:47,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38d19430[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:47,311 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:47,311 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,312 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,313 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:47,313 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:47,313 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:47,313 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,313 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:47,315 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:47,315 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:47,315 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:47,316 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:47,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:47,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:47,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:47,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:47,329 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:47,329 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:47,329 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:47,329 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,330 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:47,334 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,334 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,335 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:47,335 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:47,337 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,341 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,341 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,341 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,341 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,341 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,342 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:47,342 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:47,342 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:47,342 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:47,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:47,342 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:47,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:47,343 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:47,343 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:47,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-22 15:53:47,443 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:47,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:47,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:47,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:47,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,456 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:47,456 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:47,457 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:47,457 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:47,457 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,457 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,459 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:47,459 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:47,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:47,459 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,460 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:47,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:47,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,461 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,462 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-22 15:53:47,462 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,462 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:47,462 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:47,462 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:47,462 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:47,462 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:47,463 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:47,463 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:47,463 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:47,493 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078b78668486494f6bb5e9b99fe083cf7a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509078b78668486494f6bb5e9b99fe083cf7a 2023-03-22 15:53:47,504 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/55d7037b5bbc4b5fbdb4e2644860ab94 2023-03-22 15:53:47,514 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/55d7037b5bbc4b5fbdb4e2644860ab94 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/55d7037b5bbc4b5fbdb4e2644860ab94 2023-03-22 15:53:47,522 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/55d7037b5bbc4b5fbdb4e2644860ab94, entries=1, sequenceid=37, filesize=5.0 K 2023-03-22 15:53:47,523 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 60ms, sequenceid=37, compaction requested=true 2023-03-22 15:53:47,523 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:47,523 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:47,523 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:47,524 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:47,524 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,524 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:47,524 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:47,526 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,526 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:47,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:47,527 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,527 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:47,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:47,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:47,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:47,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:47,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7788fc4b[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:47,529 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:47,529 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,531 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:47,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:47,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:47,532 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,532 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:47,533 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,533 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:47,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:47,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:47,533 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:47,533 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:47,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:47,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:47,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:47,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:47,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:47,537 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:47,537 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:47,537 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:47,537 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,537 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:47,539 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,539 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:47,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:47,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,539 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:47,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:47,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:47,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:47,541 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:47,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,543 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:47,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,546 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:47,546 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:47,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:47,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:47,546 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:47,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:47,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,547 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:47,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-22 15:53:47,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:47,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:47,549 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:47,549 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:47,649 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:47,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:47,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:47,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:47,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,662 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:47,663 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:47,668 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-22 15:53:47,668 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-22 15:53:47,669 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,669 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,671 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,671 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:47,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:47,672 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:47,672 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:47,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:47,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,673 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,673 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-22 15:53:47,673 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:47,673 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-22 15:53:47,673 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:47,675 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-22 15:53:47,676 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:47,676 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:47,676 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:47,676 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. started... 2023-03-22 15:53:47,676 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7aa87cf5a808cbf44085918270c1463d 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:47,707 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a961245feada41b48cab4ca2e0e3e353 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a961245feada41b48cab4ca2e0e3e353 2023-03-22 15:53:48,119 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/4f02aec308ee47fa920ab96b24d4fa3c 2023-03-22 15:53:48,129 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/4f02aec308ee47fa920ab96b24d4fa3c as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4f02aec308ee47fa920ab96b24d4fa3c 2023-03-22 15:53:48,138 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4f02aec308ee47fa920ab96b24d4fa3c, entries=1, sequenceid=41, filesize=5.0 K 2023-03-22 15:53:48,139 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7aa87cf5a808cbf44085918270c1463d in 463ms, sequenceid=41, compaction requested=true 2023-03-22 15:53:48,140 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:48,140 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:48,140 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:48,140 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:48,140 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,140 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-22 15:53:48,140 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-22 15:53:48,142 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,142 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:48,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:48,143 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:48,143 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:48,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:48,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:48,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-22 15:53:48,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77ad8264[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:48,145 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-22 15:53:48,145 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,146 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,146 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-22 15:53:48,147 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-22 15:53:48,146 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,147 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:48,147 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:48,167 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-22 15:53:48,167 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,167 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:48,167 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-22 15:53:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:48,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-22 15:53:48,172 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-22 15:53:48,172 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:48,172 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:48,172 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,172 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:48,174 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,174 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:48,174 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:48,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:48,175 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,177 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:48,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:48,181 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:48,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:48,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-22 15:53:48,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-22 15:53:48,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:48,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:48,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:48,182 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:48,182 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:48,282 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:48,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:48,289 DEBUG [Listener at localhost/43289] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-22 15:53:48,293 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59272, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-22 15:53:48,294 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:48,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-22 15:53:48,311 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-22 15:53:48,313 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501028313 2023-03-22 15:53:48,313 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:53:48,314 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:53:48,319 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-22 15:53:48,319 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:48,319 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x1dc63c7f to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:53:48,328 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@8035450, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:53:48,330 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:48,881 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010304bf62f80732494eb26566651e0285cc to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010304bf62f80732494eb26566651e0285cc 2023-03-22 15:53:48,883 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:48,912 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:53:48,914 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43550, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:53:48,941 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:53:48,944 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59276, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:53:49,020 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e20af973c3034c7fb7892f799c6d120d first=Optional[r3] last=Optional[r6] 2023-03-22 15:53:49,037 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e20af973c3034c7fb7892f799c6d120d}] 2023-03-22 15:53:49,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e20af973c3034c7fb7892f799c6d120d for inclusion in 7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:49,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-22 15:53:49,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:53:49,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:49,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e20af973c3034c7fb7892f799c6d120d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__jvs9qm3153upovh4ils54hkvt8bofikq0rvtidp4vth6bs8mrp4vbuuk4omu21of/family1/e20af973c3034c7fb7892f799c6d120d 2023-03-22 15:53:49,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__jvs9qm3153upovh4ils54hkvt8bofikq0rvtidp4vth6bs8mrp4vbuuk4omu21of/family1/e20af973c3034c7fb7892f799c6d120d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/eab71538af0e4bcbb12054143bdf61df_SeqId_44_ 2023-03-22 15:53:49,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__jvs9qm3153upovh4ils54hkvt8bofikq0rvtidp4vth6bs8mrp4vbuuk4omu21of/family1/e20af973c3034c7fb7892f799c6d120d into 7aa87cf5a808cbf44085918270c1463d/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/eab71538af0e4bcbb12054143bdf61df_SeqId_44_ - updating store file list. 2023-03-22 15:53:49,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/eab71538af0e4bcbb12054143bdf61df_SeqId_44_ into 7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:49,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__jvs9qm3153upovh4ils54hkvt8bofikq0rvtidp4vth6bs8mrp4vbuuk4omu21of/family1/e20af973c3034c7fb7892f799c6d120d into 7aa87cf5a808cbf44085918270c1463d/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/eab71538af0e4bcbb12054143bdf61df_SeqId_44_) 2023-03-22 15:53:49,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__jvs9qm3153upovh4ils54hkvt8bofikq0rvtidp4vth6bs8mrp4vbuuk4omu21of/family1/e20af973c3034c7fb7892f799c6d120d 2023-03-22 15:53:49,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-22 15:53:49,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-22 15:53:49,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(1912): 7aa87cf5a808cbf44085918270c1463d/family1 is initiating minor compaction 2023-03-22 15:53:49,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@706e35f8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:53:49,101 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7aa87cf5a808cbf44085918270c1463d/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:49,102 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/3d2c91c679504914b199d14a17bb8a68, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/0e65ce90ee3c4a21bb7232b2d950fe2b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4c6f0a007a854c8a9d0991ca2f6120fd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/028506fc516f445ab7cd8f58453264e0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/084f5fa86c9d43dea09a65c36df4b8f7, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/c60fb9db965d40c69eb69825fba1e198, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/5bcc23962f43430f9264b93b474ac7d9, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/55d7037b5bbc4b5fbdb4e2644860ab94, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4f02aec308ee47fa920ab96b24d4fa3c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/eab71538af0e4bcbb12054143bdf61df_SeqId_44_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp, totalSize=50.2 K 2023-03-22 15:53:49,103 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 3d2c91c679504914b199d14a17bb8a68, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-22 15:53:49,103 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 0e65ce90ee3c4a21bb7232b2d950fe2b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-22 15:53:49,103 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 4c6f0a007a854c8a9d0991ca2f6120fd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-22 15:53:49,103 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 028506fc516f445ab7cd8f58453264e0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-22 15:53:49,103 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 084f5fa86c9d43dea09a65c36df4b8f7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-22 15:53:49,103 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting c60fb9db965d40c69eb69825fba1e198, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-22 15:53:49,103 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 5bcc23962f43430f9264b93b474ac7d9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-22 15:53:49,103 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 55d7037b5bbc4b5fbdb4e2644860ab94, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-22 15:53:49,103 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 4f02aec308ee47fa920ab96b24d4fa3c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-22 15:53:49,104 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting eab71538af0e4bcbb12054143bdf61df_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-22 15:53:49,123 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:49,125 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:53:49,132 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283db2d435a45041b88d0cb8be22ed7c37 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283db2d435a45041b88d0cb8be22ed7c37 2023-03-22 15:53:49,134 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231432cc91b70614694bcde226575077c5b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231432cc91b70614694bcde226575077c5b 2023-03-22 15:53:49,136 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101ae1ea3c0b00a4fa9bc617f20e7cbb066 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101ae1ea3c0b00a4fa9bc617f20e7cbb066 2023-03-22 15:53:49,139 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103c7d1f72810b74140ba7ce7fe547a7392 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103c7d1f72810b74140ba7ce7fe547a7392 2023-03-22 15:53:49,139 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-22 15:53:49,139 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-22 15:53:49,140 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:49,140 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:49,140 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:49,141 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:49,141 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:49,141 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:49,141 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:49,141 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:49,210 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7aa87cf5a808cbf44085918270c1463d#family1#compaction#10 average throughput is 0.16 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:53:49,676 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/22b1e75b9c334b5d8183de2b82a008b2 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/22b1e75b9c334b5d8183de2b82a008b2 2023-03-22 15:53:49,697 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 7aa87cf5a808cbf44085918270c1463d/family1 of 7aa87cf5a808cbf44085918270c1463d into 22b1e75b9c334b5d8183de2b82a008b2(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:53:49,697 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:49,699 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1679500429099; duration=0sec 2023-03-22 15:53:49,700 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:53:49,803 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/3d2c91c679504914b199d14a17bb8a68, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/0e65ce90ee3c4a21bb7232b2d950fe2b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4c6f0a007a854c8a9d0991ca2f6120fd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/028506fc516f445ab7cd8f58453264e0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/084f5fa86c9d43dea09a65c36df4b8f7, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/c60fb9db965d40c69eb69825fba1e198, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/5bcc23962f43430f9264b93b474ac7d9, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/55d7037b5bbc4b5fbdb4e2644860ab94, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4f02aec308ee47fa920ab96b24d4fa3c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/eab71538af0e4bcbb12054143bdf61df_SeqId_44_] to archive 2023-03-22 15:53:49,805 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:53:49,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/3d2c91c679504914b199d14a17bb8a68 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/3d2c91c679504914b199d14a17bb8a68 2023-03-22 15:53:49,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/0e65ce90ee3c4a21bb7232b2d950fe2b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/0e65ce90ee3c4a21bb7232b2d950fe2b 2023-03-22 15:53:49,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4c6f0a007a854c8a9d0991ca2f6120fd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4c6f0a007a854c8a9d0991ca2f6120fd 2023-03-22 15:53:49,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/028506fc516f445ab7cd8f58453264e0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/028506fc516f445ab7cd8f58453264e0 2023-03-22 15:53:49,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/084f5fa86c9d43dea09a65c36df4b8f7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/084f5fa86c9d43dea09a65c36df4b8f7 2023-03-22 15:53:49,819 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/c60fb9db965d40c69eb69825fba1e198 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/c60fb9db965d40c69eb69825fba1e198 2023-03-22 15:53:49,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/5bcc23962f43430f9264b93b474ac7d9 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/5bcc23962f43430f9264b93b474ac7d9 2023-03-22 15:53:49,822 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/55d7037b5bbc4b5fbdb4e2644860ab94 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/55d7037b5bbc4b5fbdb4e2644860ab94 2023-03-22 15:53:49,825 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4f02aec308ee47fa920ab96b24d4fa3c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4f02aec308ee47fa920ab96b24d4fa3c 2023-03-22 15:53:49,827 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/eab71538af0e4bcbb12054143bdf61df_SeqId_44_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/eab71538af0e4bcbb12054143bdf61df_SeqId_44_ 2023-03-22 15:53:50,121 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050368446a19974e82913f8aab07f6205e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512050368446a19974e82913f8aab07f6205e 2023-03-22 15:53:50,123 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:50,159 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7be2d25e4ca043b69d4da0b41f52ca95 first=Optional[r00] last=Optional[r2] 2023-03-22 15:53:50,160 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7be2d25e4ca043b69d4da0b41f52ca95}] 2023-03-22 15:53:50,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7be2d25e4ca043b69d4da0b41f52ca95 for inclusion in 7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:50,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-22 15:53:50,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:53:50,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:50,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7be2d25e4ca043b69d4da0b41f52ca95 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6ic6it057focpb9c16gnvjl15ihlqh1jrsf0hg6irql6cahb57n0o46q7eq0h4kf/family1/7be2d25e4ca043b69d4da0b41f52ca95 2023-03-22 15:53:50,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6ic6it057focpb9c16gnvjl15ihlqh1jrsf0hg6irql6cahb57n0o46q7eq0h4kf/family1/7be2d25e4ca043b69d4da0b41f52ca95 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/2c486670d95c4a68ab59916214166a2d_SeqId_47_ 2023-03-22 15:53:50,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6ic6it057focpb9c16gnvjl15ihlqh1jrsf0hg6irql6cahb57n0o46q7eq0h4kf/family1/7be2d25e4ca043b69d4da0b41f52ca95 into 7aa87cf5a808cbf44085918270c1463d/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/2c486670d95c4a68ab59916214166a2d_SeqId_47_ - updating store file list. 2023-03-22 15:53:50,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/2c486670d95c4a68ab59916214166a2d_SeqId_47_ into 7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:50,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6ic6it057focpb9c16gnvjl15ihlqh1jrsf0hg6irql6cahb57n0o46q7eq0h4kf/family1/7be2d25e4ca043b69d4da0b41f52ca95 into 7aa87cf5a808cbf44085918270c1463d/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/2c486670d95c4a68ab59916214166a2d_SeqId_47_) 2023-03-22 15:53:50,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__6ic6it057focpb9c16gnvjl15ihlqh1jrsf0hg6irql6cahb57n0o46q7eq0h4kf/family1/7be2d25e4ca043b69d4da0b41f52ca95 2023-03-22 15:53:50,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-22 15:53:50,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-22 15:53:50,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(1912): 7aa87cf5a808cbf44085918270c1463d/family1 is initiating minor compaction (all files) 2023-03-22 15:53:50,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@240f714; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:53:50,185 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7aa87cf5a808cbf44085918270c1463d/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:50,186 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4486805946524859b3e050e19e3798b2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/22b1e75b9c334b5d8183de2b82a008b2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/2c486670d95c4a68ab59916214166a2d_SeqId_47_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp, totalSize=358.6 K 2023-03-22 15:53:50,187 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 4486805946524859b3e050e19e3798b2, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840482 2023-03-22 15:53:50,187 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 22b1e75b9c334b5d8183de2b82a008b2, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240482 2023-03-22 15:53:50,188 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 2c486670d95c4a68ab59916214166a2d_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840482 2023-03-22 15:53:50,189 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:50,190 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:53:50,193 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511300b364bc4d3e04718b0b7b18cf5ffcfc7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511300b364bc4d3e04718b0b7b18cf5ffcfc7 2023-03-22 15:53:50,196 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201fe445a5829c84cb3af67d69be94197b9 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201fe445a5829c84cb3af67d69be94197b9 2023-03-22 15:53:50,198 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512059c7c5673531e470093605fdf5909b07e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512059c7c5673531e470093605fdf5909b07e 2023-03-22 15:53:50,198 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-22 15:53:50,198 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-22 15:53:50,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:50,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:50,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:53:50,201 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-22 15:53:50,204 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.9010 sec 2023-03-22 15:53:50,260 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7aa87cf5a808cbf44085918270c1463d#family1#compaction#11 average throughput is 3.75 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:53:50,374 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/.tmp/family1/308e567baf554709b27cf569225ff162 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/308e567baf554709b27cf569225ff162 2023-03-22 15:53:50,387 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 7aa87cf5a808cbf44085918270c1463d/family1 of 7aa87cf5a808cbf44085918270c1463d into 308e567baf554709b27cf569225ff162(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:53:50,387 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:50,388 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1679500430185; duration=0sec 2023-03-22 15:53:50,388 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:53:50,400 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4486805946524859b3e050e19e3798b2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/22b1e75b9c334b5d8183de2b82a008b2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/2c486670d95c4a68ab59916214166a2d_SeqId_47_] to archive 2023-03-22 15:53:50,402 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:53:50,404 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4486805946524859b3e050e19e3798b2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/4486805946524859b3e050e19e3798b2 2023-03-22 15:53:50,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/22b1e75b9c334b5d8183de2b82a008b2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/22b1e75b9c334b5d8183de2b82a008b2 2023-03-22 15:53:50,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/2c486670d95c4a68ab59916214166a2d_SeqId_47_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/2c486670d95c4a68ab59916214166a2d_SeqId_47_ 2023-03-22 15:53:50,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-22 15:53:50,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:50,915 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-22 15:53:50,918 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-22 15:53:50,918 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-22 15:53:50,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-22 15:53:50,939 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=7aa87cf5a808cbf44085918270c1463d, REOPEN/MOVE}] 2023-03-22 15:53:50,940 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=7aa87cf5a808cbf44085918270c1463d, REOPEN/MOVE 2023-03-22 15:53:50,942 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=7aa87cf5a808cbf44085918270c1463d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:50,942 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679500430941"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500430941"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500430941"}]},"ts":"1679500430941"} 2023-03-22 15:53:50,945 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 7aa87cf5a808cbf44085918270c1463d, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:53:51,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:51,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7aa87cf5a808cbf44085918270c1463d, disabling compactions & flushes 2023-03-22 15:53:51,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:51,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:51,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. after waiting 0 ms 2023-03-22 15:53:51,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:51,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-22 15:53:51,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:51,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:51,124 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 7aa87cf5a808cbf44085918270c1463d to self. 2023-03-22 15:53:51,132 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:51,133 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=7aa87cf5a808cbf44085918270c1463d, regionState=CLOSED 2023-03-22 15:53:51,133 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679500431133"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500431133"}]},"ts":"1679500431133"} 2023-03-22 15:53:51,138 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-22 15:53:51,139 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 7aa87cf5a808cbf44085918270c1463d, server=jenkins-hbase4.apache.org,38637,1679500416185 in 190 msec 2023-03-22 15:53:51,140 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=7aa87cf5a808cbf44085918270c1463d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=true 2023-03-22 15:53:51,291 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=7aa87cf5a808cbf44085918270c1463d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:51,291 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679500431291"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500431291"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500431291"}]},"ts":"1679500431291"} 2023-03-22 15:53:51,294 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 7aa87cf5a808cbf44085918270c1463d, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:53:51,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:51,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7aa87cf5a808cbf44085918270c1463d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:53:51,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:51,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:51,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:51,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:51,454 INFO [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:51,455 DEBUG [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:51,455 DEBUG [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:51,455 INFO [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7aa87cf5a808cbf44085918270c1463d columnFamilyName family1 2023-03-22 15:53:51,464 DEBUG [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/308e567baf554709b27cf569225ff162 2023-03-22 15:53:51,464 INFO [StoreOpener-7aa87cf5a808cbf44085918270c1463d-1] regionserver.HStore(310): Store=7aa87cf5a808cbf44085918270c1463d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:51,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:51,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:51,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:53:51,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7aa87cf5a808cbf44085918270c1463d; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72911070, jitterRate=0.08645960688591003}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:53:51,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:51,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d., pid=17, masterSystemTime=1679500431447 2023-03-22 15:53:51,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:51,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:53:51,476 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=7aa87cf5a808cbf44085918270c1463d, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:51,477 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679500431476"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500431476"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500431476"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500431476"}]},"ts":"1679500431476"} 2023-03-22 15:53:51,482 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-22 15:53:51,482 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 7aa87cf5a808cbf44085918270c1463d, server=jenkins-hbase4.apache.org,38637,1679500416185 in 185 msec 2023-03-22 15:53:51,485 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-22 15:53:51,485 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=7aa87cf5a808cbf44085918270c1463d, REOPEN/MOVE in 543 msec 2023-03-22 15:53:51,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-22 15:53:51,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 563 msec 2023-03-22 15:53:51,492 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0080 sec 2023-03-22 15:53:51,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-22 15:53:51,504 INFO [Listener at localhost/43289] compactions.TestMobCompactor(278): alter status finished 2023-03-22 15:53:51,506 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-22 15:53:51,508 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-22 15:53:51,509 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-22 15:53:51,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:53:51,510 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501031510 2023-03-22 15:53:51,511 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:53:51,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-22 15:53:51,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:51,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x621d1842 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:53:51,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@f1685ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:53:51,527 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:51,527 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:51,527 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:51,527 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:51,527 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:52,011 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112894647806103e438780e4b57e0df324d1 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112894647806103e438780e4b57e0df324d1 2023-03-22 15:53:52,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:52,030 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:53:52,032 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43558, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:53:52,049 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:53:52,053 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59286, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:53:52,083 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5bd1634209104f33ae9b64ebea675b26 first=Optional[r05] last=Optional[r8] 2023-03-22 15:53:52,084 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5bd1634209104f33ae9b64ebea675b26}] 2023-03-22 15:53:52,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5bd1634209104f33ae9b64ebea675b26 for inclusion in 7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:52,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-22 15:53:52,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:53:52,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:53:52,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5bd1634209104f33ae9b64ebea675b26 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k0sh3tugqb4tigbtvnqburoga2nujm4c5hnjfs77jkl6gq5bontjue2cm30terhg/family1/5bd1634209104f33ae9b64ebea675b26 2023-03-22 15:53:52,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k0sh3tugqb4tigbtvnqburoga2nujm4c5hnjfs77jkl6gq5bontjue2cm30terhg/family1/5bd1634209104f33ae9b64ebea675b26 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/70fe6481fa5a40ccae99ac42866b1961_SeqId_53_ 2023-03-22 15:53:52,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k0sh3tugqb4tigbtvnqburoga2nujm4c5hnjfs77jkl6gq5bontjue2cm30terhg/family1/5bd1634209104f33ae9b64ebea675b26 into 7aa87cf5a808cbf44085918270c1463d/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/70fe6481fa5a40ccae99ac42866b1961_SeqId_53_ - updating store file list. 2023-03-22 15:53:52,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/70fe6481fa5a40ccae99ac42866b1961_SeqId_53_ into 7aa87cf5a808cbf44085918270c1463d/family1 2023-03-22 15:53:52,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k0sh3tugqb4tigbtvnqburoga2nujm4c5hnjfs77jkl6gq5bontjue2cm30terhg/family1/5bd1634209104f33ae9b64ebea675b26 into 7aa87cf5a808cbf44085918270c1463d/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/family1/70fe6481fa5a40ccae99ac42866b1961_SeqId_53_) 2023-03-22 15:53:52,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k0sh3tugqb4tigbtvnqburoga2nujm4c5hnjfs77jkl6gq5bontjue2cm30terhg/family1/5bd1634209104f33ae9b64ebea675b26 2023-03-22 15:53:52,116 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:52,117 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:53:52,120 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a961245feada41b48cab4ca2e0e3e353 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a961245feada41b48cab4ca2e0e3e353 2023-03-22 15:53:52,121 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112846552b3c0a9f433d9c65a96e6306e720 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112846552b3c0a9f433d9c65a96e6306e720 2023-03-22 15:53:52,121 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-22 15:53:52,121 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-22 15:53:52,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:52,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:52,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-22 15:53:52,122 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-22 15:53:52,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:53:52,124 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-22 15:53:52,131 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 617 msec 2023-03-22 15:53:52,252 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=248 (was 209) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58225@0x1dc63c7f sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1580181509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xca54f77-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36512 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36396 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36468 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:38637-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36482 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36284 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2054959231_15 at /127.0.0.1:36540 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58225@0x621d1842-SendThread(127.0.0.1:58225) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x2fd44b49-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58225@0x621d1842-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58225@0x621d1842 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1580181509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36366 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xca54f77-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58225@0x1dc63c7f-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36258 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/cluster_73db91c0-5f76-0b43-c651-cc63b18b2aa1/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36388 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36594 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36410 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36536 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36304 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xca54f77-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36496 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/cluster_73db91c0-5f76-0b43-c651-cc63b18b2aa1/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58225@0x1dc63c7f-SendThread(127.0.0.1:58225) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36340 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36382 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2fd44b49-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=599 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=437 (was 295) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=7004 (was 7434) 2023-03-22 15:53:52,293 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=599, MaxFileDescriptor=60000, SystemLoadAverage=437, ProcessCount=172, AvailableMemoryMB=7003 2023-03-22 15:53:52,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-22 15:53:52,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:52,304 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:53:52,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-22 15:53:52,305 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:53:52,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-22 15:53:52,308 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:53:52,309 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132 empty. 2023-03-22 15:53:52,311 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:53:52,311 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-22 15:53:52,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-22 15:53:52,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-22 15:53:52,746 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-22 15:53:52,748 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => d05b1269482a465dff6cf06fb1c98132, NAME => 'testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:53:52,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-22 15:53:53,167 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:53,167 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing d05b1269482a465dff6cf06fb1c98132, disabling compactions & flushes 2023-03-22 15:53:53,167 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:53,167 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:53,167 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. after waiting 0 ms 2023-03-22 15:53:53,167 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:53,167 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:53,167 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:53:53,170 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:53:53,172 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679500433171"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500433171"}]},"ts":"1679500433171"} 2023-03-22 15:53:53,173 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-22 15:53:53,174 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:53:53,175 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500433175"}]},"ts":"1679500433175"} 2023-03-22 15:53:53,176 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-22 15:53:53,179 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=d05b1269482a465dff6cf06fb1c98132, ASSIGN}] 2023-03-22 15:53:53,181 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=d05b1269482a465dff6cf06fb1c98132, ASSIGN 2023-03-22 15:53:53,182 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=d05b1269482a465dff6cf06fb1c98132, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:53:53,333 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=d05b1269482a465dff6cf06fb1c98132, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:53,334 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679500433333"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500433333"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500433333"}]},"ts":"1679500433333"} 2023-03-22 15:53:53,336 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure d05b1269482a465dff6cf06fb1c98132, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:53:53,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-22 15:53:53,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:53,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d05b1269482a465dff6cf06fb1c98132, NAME => 'testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:53:53,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:53:53,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:53:53,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:53:53,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:53:53,495 INFO [StoreOpener-d05b1269482a465dff6cf06fb1c98132-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:53:53,496 DEBUG [StoreOpener-d05b1269482a465dff6cf06fb1c98132-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1 2023-03-22 15:53:53,497 DEBUG [StoreOpener-d05b1269482a465dff6cf06fb1c98132-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1 2023-03-22 15:53:53,497 INFO [StoreOpener-d05b1269482a465dff6cf06fb1c98132-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d05b1269482a465dff6cf06fb1c98132 columnFamilyName family1 2023-03-22 15:53:53,499 INFO [StoreOpener-d05b1269482a465dff6cf06fb1c98132-1] regionserver.HStore(310): Store=d05b1269482a465dff6cf06fb1c98132/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:53:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:53:53,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:53:53,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:53:53,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:53:53,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d05b1269482a465dff6cf06fb1c98132; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62675674, jitterRate=-0.06605967879295349}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:53:53,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:53:53,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132., pid=21, masterSystemTime=1679500433489 2023-03-22 15:53:53,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:53,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:53,511 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=d05b1269482a465dff6cf06fb1c98132, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:53,511 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679500433511"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500433511"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500433511"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500433511"}]},"ts":"1679500433511"} 2023-03-22 15:53:53,515 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-22 15:53:53,516 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure d05b1269482a465dff6cf06fb1c98132, server=jenkins-hbase4.apache.org,38637,1679500416185 in 177 msec 2023-03-22 15:53:53,518 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-22 15:53:53,518 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=d05b1269482a465dff6cf06fb1c98132, ASSIGN in 337 msec 2023-03-22 15:53:53,519 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:53:53,519 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500433519"}]},"ts":"1679500433519"} 2023-03-22 15:53:53,520 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-22 15:53:53,523 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:53:53,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.2230 sec 2023-03-22 15:53:54,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-22 15:53:54,415 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-22 15:53:54,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:54,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,462 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:54,462 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:54,463 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:53:54,463 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:53:54,464 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,464 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,466 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:54,466 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:54,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:54,466 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,466 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:54,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,467 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,467 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,468 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,468 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:53:54,468 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:54,468 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:53:54,472 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:54,472 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:54,472 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:54,472 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:53:54,473 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-22 15:53:54,516 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511301288c1487edb4c479ed649feaa398d10 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511301288c1487edb4c479ed649feaa398d10 2023-03-22 15:53:54,521 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/0bf968c846c34d6fa976fd5f699baa02 2023-03-22 15:53:54,528 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/0bf968c846c34d6fa976fd5f699baa02 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/0bf968c846c34d6fa976fd5f699baa02 2023-03-22 15:53:54,535 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/0bf968c846c34d6fa976fd5f699baa02, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-22 15:53:54,536 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 63ms, sequenceid=5, compaction requested=false 2023-03-22 15:53:54,536 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:54,537 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:53:54,537 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:54,537 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:54,537 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:54,538 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,538 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:53:54,538 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:53:54,540 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:54,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:54,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:54,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:54,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:54,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:54,542 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:53:54,542 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60213d0d[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:54,542 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:53:54,542 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,546 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:53:54,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:53:54,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:54,546 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,546 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:54,548 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:53:54,548 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:54,548 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:53:54,548 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:54,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:54,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:54,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:54,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:54,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:54,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:53:54,556 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:53:54,556 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:54,556 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:54,556 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,556 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:54,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:54,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:54,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:54,559 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:54,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:54,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:54,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:54,561 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:54,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,565 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:54,565 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:54,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:54,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:54,566 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,566 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,566 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,566 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,566 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,566 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:54,566 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:54,566 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:54,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:54,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:54,567 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:54,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:53:54,568 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:54,668 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:54,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:54,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:54,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:54,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,677 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:54,678 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:54,678 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:53:54,679 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:53:54,679 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,679 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,682 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,682 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:54,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:54,682 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:54,682 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,684 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:53:54,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:54,684 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:53:54,684 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:54,685 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:53:54,685 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:54,685 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:54,685 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:54,685 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:53:54,685 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-22 15:53:55,107 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112810dfee70739b4c918f343d2be5c1243a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112810dfee70739b4c918f343d2be5c1243a 2023-03-22 15:53:55,514 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/7b48444d3a09404daf7358c9e2b6e952 2023-03-22 15:53:55,523 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/7b48444d3a09404daf7358c9e2b6e952 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/7b48444d3a09404daf7358c9e2b6e952 2023-03-22 15:53:55,529 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/7b48444d3a09404daf7358c9e2b6e952, entries=1, sequenceid=9, filesize=5.0 K 2023-03-22 15:53:55,530 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 845ms, sequenceid=9, compaction requested=false 2023-03-22 15:53:55,531 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:53:55,531 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:53:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:53:55,533 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:55,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:55,533 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:55,534 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,534 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:55,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:55,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:53:55,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5465552d[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:55,536 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:53:55,536 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,537 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,537 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:53:55,537 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:53:55,538 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:55,538 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,538 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:53:55,540 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:55,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:55,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:53:55,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:55,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:55,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:55,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:55,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:53:55,544 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:53:55,544 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:55,544 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:55,544 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,544 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:55,548 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,548 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:55,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,548 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:55,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:55,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:55,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:55,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,549 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:55,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,551 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:55,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:55,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:55,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:55,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,557 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:55,557 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:55,557 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:55,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:55,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:55,558 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:55,559 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:55,659 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:55,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:55,666 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:55,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:55,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,668 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:55,668 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:55,669 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:53:55,669 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:53:55,669 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,669 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,671 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:55,671 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:55,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:55,671 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:55,671 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:55,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:55,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,672 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,673 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:53:55,673 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:53:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:53:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:55,673 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:55,673 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:55,674 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:53:55,674 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:56,093 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201452550d91a974c938bbbc9ac1767af55 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201452550d91a974c938bbbc9ac1767af55 2023-03-22 15:53:56,501 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/2616a8adec1241308e9aaa7e14929c45 2023-03-22 15:53:56,509 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/2616a8adec1241308e9aaa7e14929c45 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/2616a8adec1241308e9aaa7e14929c45 2023-03-22 15:53:56,516 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/2616a8adec1241308e9aaa7e14929c45, entries=1, sequenceid=13, filesize=4.9 K 2023-03-22 15:53:56,517 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 843ms, sequenceid=13, compaction requested=true 2023-03-22 15:53:56,517 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:53:56,517 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:56,517 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:56,517 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:56,517 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,517 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:53:56,518 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:53:56,519 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,519 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:56,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:56,520 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:56,520 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:56,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:56,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:56,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:53:56,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@643f914e[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:56,522 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:53:56,522 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,524 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,525 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:53:56,525 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:53:56,525 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,525 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:56,525 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:56,526 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:53:56,527 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,527 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:56,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:56,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:56,527 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:53:56,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:56,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:56,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:56,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:56,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:53:56,534 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:53:56,534 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:56,535 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:56,535 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,535 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:56,536 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,536 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,536 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:56,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:56,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:56,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:56,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:56,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:56,537 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:56,539 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:56,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,542 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:56,542 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:56,542 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:56,542 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,542 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:56,542 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:56,543 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:56,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:53:56,543 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:56,543 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,543 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:56,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:56,544 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:56,544 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:56,644 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:56,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:56,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:56,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:56,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,653 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:56,653 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:56,654 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:53:56,654 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:53:56,654 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,655 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,657 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:56,657 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:56,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:56,658 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:56,658 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:56,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,658 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:56,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,659 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,659 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:53:56,659 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:56,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:53:56,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:56,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:53:56,660 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:56,660 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:56,660 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:56,660 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:53:56,660 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:56,664 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:53:56,703 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056f66d0682d454e43a3b3e226bba54778 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512056f66d0682d454e43a3b3e226bba54778 2023-03-22 15:53:57,119 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/f6eeb6713c0549aa89c79b457674a121 2023-03-22 15:53:57,129 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/f6eeb6713c0549aa89c79b457674a121 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/f6eeb6713c0549aa89c79b457674a121 2023-03-22 15:53:57,135 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/f6eeb6713c0549aa89c79b457674a121, entries=1, sequenceid=17, filesize=4.9 K 2023-03-22 15:53:57,137 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 477ms, sequenceid=17, compaction requested=true 2023-03-22 15:53:57,137 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:53:57,137 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:57,137 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:57,137 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:57,137 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,137 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:53:57,137 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:53:57,139 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,139 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:57,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:57,140 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,140 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:57,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:57,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:57,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:57,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:53:57,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f1a810f[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:57,142 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:53:57,142 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,145 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,145 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:53:57,145 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:53:57,145 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:57,145 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,145 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:57,147 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,147 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:53:57,147 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:57,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:57,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:57,147 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:53:57,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:57,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:57,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:57,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:57,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:53:57,152 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:53:57,152 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:57,152 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:57,152 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,152 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:57,155 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,155 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:57,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:57,155 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:57,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:57,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:57,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:57,156 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:57,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,158 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:57,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,161 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:57,161 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:57,161 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,161 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,161 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,161 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,161 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,161 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,161 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,161 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:57,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:57,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:57,163 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:57,163 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:57,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:53:57,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:57,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:57,263 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:57,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:57,270 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:57,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:57,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,273 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:57,273 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:57,274 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:53:57,274 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:53:57,274 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,274 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,276 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,276 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:57,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:57,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:57,276 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,276 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:57,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,278 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:53:57,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,278 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:53:57,278 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:57,278 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:53:57,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:57,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:57,279 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:57,279 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:53:57,279 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:57,709 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122890dfcc7329ba40c6baeec4dfa819f749 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122890dfcc7329ba40c6baeec4dfa819f749 2023-03-22 15:53:57,717 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/5b0711e3cb3f4e5e9beef46a4bd70a48 2023-03-22 15:53:57,723 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/5b0711e3cb3f4e5e9beef46a4bd70a48 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/5b0711e3cb3f4e5e9beef46a4bd70a48 2023-03-22 15:53:57,729 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/5b0711e3cb3f4e5e9beef46a4bd70a48, entries=1, sequenceid=21, filesize=4.9 K 2023-03-22 15:53:57,730 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 451ms, sequenceid=21, compaction requested=true 2023-03-22 15:53:57,731 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:53:57,731 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:57,731 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:57,731 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:57,731 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,731 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:53:57,731 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:53:57,808 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,808 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:57,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:57,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:57,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:57,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:57,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:53:57,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16ba02b4[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:57,811 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:53:57,811 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,862 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,863 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:53:57,863 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:53:57,863 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:57,863 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,863 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:57,931 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,931 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:53:57,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:57,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:57,931 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:57,932 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:53:57,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:57,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:57,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:57,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:57,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:57,935 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:53:57,935 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:53:57,935 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:57,935 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:57,935 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:57,935 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:58,100 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,100 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:58,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:58,101 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:58,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:58,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:58,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:58,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,102 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,103 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,163 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:58,163 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:58,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:58,163 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,163 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,163 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,163 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,163 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,163 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:58,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:58,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:58,163 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:58,163 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,164 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:58,164 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:58,265 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:58,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:58,273 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:58,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:58,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,275 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:58,275 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:58,276 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:53:58,276 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:53:58,276 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,277 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,362 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:58,362 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:58,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:58,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,363 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,363 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:58,363 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:58,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,364 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,364 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:53:58,364 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,364 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:53:58,364 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:58,365 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:53:58,365 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:58,365 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:58,365 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:58,365 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:53:58,365 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:58,782 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123169ff0794ac44479599b858a8059b6d21 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123169ff0794ac44479599b858a8059b6d21 2023-03-22 15:53:58,788 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/14b7856f55964796834e8b6bbbe79250 2023-03-22 15:53:58,796 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/14b7856f55964796834e8b6bbbe79250 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/14b7856f55964796834e8b6bbbe79250 2023-03-22 15:53:58,802 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/14b7856f55964796834e8b6bbbe79250, entries=1, sequenceid=25, filesize=4.9 K 2023-03-22 15:53:58,803 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 438ms, sequenceid=25, compaction requested=true 2023-03-22 15:53:58,803 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:53:58,803 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:58,803 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:58,803 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:58,803 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,803 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:53:58,803 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:53:58,839 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,839 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:58,839 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,840 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:58,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:53:58,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6808e173[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:58,841 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:53:58,841 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,901 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,905 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:53:58,905 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:53:58,905 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:58,931 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:53:58,931 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:58,931 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:58,932 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:53:58,933 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:53:58,933 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:58,933 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:53:58,933 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:58,933 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:58,933 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:58,933 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:59,008 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,008 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,008 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:59,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,009 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:59,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:59,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,012 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:59,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,056 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,056 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:59,056 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:59,056 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:59,056 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:59,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:59,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:59,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:53:59,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:59,057 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,057 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,057 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,057 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,057 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,057 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:59,057 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:59,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:59,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:59,158 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:59,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:59,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:59,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:59,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,166 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:59,166 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:59,167 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:53:59,167 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:53:59,167 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,167 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,239 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,239 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:59,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:59,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:59,239 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,240 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:59,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:59,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,240 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,241 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:53:59,241 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:53:59,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:59,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:53:59,242 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:59,242 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:59,242 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:59,242 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:53:59,242 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:59,261 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e84a1a53270c4714b3e6d11cf8ab21df to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101e84a1a53270c4714b3e6d11cf8ab21df 2023-03-22 15:53:59,271 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/e5754516a20e4a10bc5dcaff09713660 2023-03-22 15:53:59,277 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/e5754516a20e4a10bc5dcaff09713660 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/e5754516a20e4a10bc5dcaff09713660 2023-03-22 15:53:59,282 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/e5754516a20e4a10bc5dcaff09713660, entries=1, sequenceid=29, filesize=4.9 K 2023-03-22 15:53:59,284 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 42ms, sequenceid=29, compaction requested=true 2023-03-22 15:53:59,284 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:53:59,284 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:59,284 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:53:59,284 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:53:59,284 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,284 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:53:59,284 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:53:59,347 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,347 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:59,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:59,347 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,347 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:53:59,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:59,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:59,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:59,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:53:59,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7025a8dd[Count = 0] remaining members to acquire global barrier 2023-03-22 15:53:59,349 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:53:59,349 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,439 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,439 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:53:59,439 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,439 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:53:59,439 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:53:59,439 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:53:59,462 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,462 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:53:59,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:59,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:59,462 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:59,463 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:53:59,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:59,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:59,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:59,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:53:59,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:53:59,466 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:53:59,467 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:53:59,467 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:53:59,467 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,467 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:53:59,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:59,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:53:59,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:53:59,548 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:53:59,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:59,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:59,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:53:59,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,551 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:53:59,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,608 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,608 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:59,608 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:53:59,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:59,608 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:53:59,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:59,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:53:59,608 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:59,609 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,609 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,609 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,609 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,609 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,609 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:53:59,610 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:53:59,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:53:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:53:59,710 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:53:59,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:53:59,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:53:59,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:53:59,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,730 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:53:59,730 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:59,730 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:53:59,730 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:53:59,731 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,731 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,793 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:53:59,793 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:53:59,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:53:59,794 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:53:59,794 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:53:59,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:53:59,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,795 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:53:59,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:53:59,795 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:53:59,795 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:53:59,796 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:53:59,797 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:53:59,797 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:53:59,797 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:53:59,797 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:53:59,797 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:53:59,819 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031a0b221455384b8aabe8f1814e6cad17 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601031a0b221455384b8aabe8f1814e6cad17 2023-03-22 15:54:00,226 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/05c9fe509c4b45c0a4459ad81717661d 2023-03-22 15:54:00,234 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/05c9fe509c4b45c0a4459ad81717661d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/05c9fe509c4b45c0a4459ad81717661d 2023-03-22 15:54:00,240 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/05c9fe509c4b45c0a4459ad81717661d, entries=1, sequenceid=33, filesize=4.9 K 2023-03-22 15:54:00,241 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 444ms, sequenceid=33, compaction requested=true 2023-03-22 15:54:00,241 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:54:00,241 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:54:00,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:00,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:00,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:54:00,241 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:54:00,301 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,301 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:00,301 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,301 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:00,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:00,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:00,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:54:00,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b1c4d2f[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:00,303 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:00,303 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,385 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,385 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:00,385 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:54:00,385 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,386 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:00,386 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:00,517 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,517 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:54:00,517 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:00,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:00,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:00,517 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:54:00,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:00,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:00,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:00,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:00,529 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:54:00,529 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:00,529 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:00,529 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,529 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:00,593 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,593 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:00,593 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:00,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:00,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,594 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:00,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:00,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:00,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:00,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,595 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:00,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,654 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:00,654 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,655 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:00,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:00,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:00,655 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:00,655 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,656 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,656 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,656 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:54:00,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:54:00,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:00,656 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,656 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:00,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:00,656 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:00,656 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:00,757 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:00,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:00,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:00,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:00,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,765 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:54:00,765 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:00,766 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:54:00,766 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:00,766 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,766 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,839 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:00,839 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:00,840 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:00,840 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:00,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,840 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:00,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,841 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,841 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:54:00,841 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:00,841 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:54:00,841 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:00,841 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:54:00,842 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:00,842 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:00,842 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:54:00,842 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:54:00,842 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:00,857 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907824b752f147c4a6bba28251ab426d6c5 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907824b752f147c4a6bba28251ab426d6c5 2023-03-22 15:54:01,263 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/a22b36f8a8ae4523a9df45442f08fcdd 2023-03-22 15:54:01,270 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/a22b36f8a8ae4523a9df45442f08fcdd as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a22b36f8a8ae4523a9df45442f08fcdd 2023-03-22 15:54:01,276 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a22b36f8a8ae4523a9df45442f08fcdd, entries=1, sequenceid=37, filesize=4.9 K 2023-03-22 15:54:01,278 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 436ms, sequenceid=37, compaction requested=true 2023-03-22 15:54:01,278 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:54:01,278 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:54:01,278 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:01,278 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:01,278 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,278 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:54:01,278 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:54:01,324 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,324 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:01,324 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,324 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:01,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:54:01,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1df54b5a[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:01,326 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:01,326 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,363 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,363 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:01,363 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:54:01,363 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:01,363 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,363 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:01,378 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:54:01,378 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,378 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:01,378 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:54:01,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:01,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:01,381 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:01,381 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:54:01,381 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:01,381 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:01,381 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,381 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:01,447 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,447 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,447 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:01,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:01,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:01,448 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:01,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:01,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,449 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:01,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,509 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:01,509 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:01,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:01,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:54:01,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:01,510 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:01,510 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:01,510 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,511 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,511 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,511 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,511 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,511 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,511 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:01,511 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:01,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:01,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:01,611 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:01,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:01,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:01,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:01,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,619 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:54:01,619 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:01,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:54:01,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:01,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,662 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:01,662 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:01,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:01,662 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:01,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,663 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:01,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:01,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,663 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:54:01,664 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:01,664 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:54:01,664 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:01,665 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:54:01,665 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:01,665 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:01,665 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:54:01,665 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. started... 2023-03-22 15:54:01,665 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d05b1269482a465dff6cf06fb1c98132 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:02,083 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a9d6ec1ee826459fb7c4fbf52e75558a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120a9d6ec1ee826459fb7c4fbf52e75558a 2023-03-22 15:54:02,489 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/a9e47df789af41e399e3c7aaa55507c4 2023-03-22 15:54:02,495 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/a9e47df789af41e399e3c7aaa55507c4 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a9e47df789af41e399e3c7aaa55507c4 2023-03-22 15:54:02,500 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a9e47df789af41e399e3c7aaa55507c4, entries=1, sequenceid=41, filesize=4.9 K 2023-03-22 15:54:02,501 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d05b1269482a465dff6cf06fb1c98132 in 836ms, sequenceid=41, compaction requested=true 2023-03-22 15:54:02,501 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:54:02,501 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:54:02,501 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:02,502 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:02,502 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,502 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:54:02,502 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:54:02,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,562 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:02,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:02,563 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:02,563 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:02,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:02,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:02,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:54:02,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@325932ee[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:02,564 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:02,564 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,585 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,585 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:02,585 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,585 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:02,585 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:54:02,585 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:02,616 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,616 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:54:02,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:02,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:02,616 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:02,616 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:54:02,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:02,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:02,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:02,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:02,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:02,619 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:54:02,619 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:02,619 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:02,619 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,619 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:02,631 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,631 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:02,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:02,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,631 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:02,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:02,632 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,633 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,639 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,639 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:02,639 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:02,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-22 15:54:02,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:02,639 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,639 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:02,639 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:02,639 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:02,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:02,640 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,641 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:02,641 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:02,641 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,641 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:02,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:02,741 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:02,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:02,743 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-22 15:54:02,745 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:54:02,746 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:54:02,747 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:02,747 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501042747 2023-03-22 15:54:02,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:54:02,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-22 15:54:02,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:54:02,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x1dbf9cfd to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:02,757 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@26bf490a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:02,757 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:54:03,596 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103966c8d952df9472ca432734f4d8269c7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103966c8d952df9472ca432734f4d8269c7 2023-03-22 15:54:03,597 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:54:03,610 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:54:03,611 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33898, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:54:03,619 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:54:03,620 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39896, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:54:03,631 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/19ac8b6eb4cf42558eb6d0ea4bd99635 first=Optional[r5] last=Optional[r6] 2023-03-22 15:54:03,632 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/19ac8b6eb4cf42558eb6d0ea4bd99635}] 2023-03-22 15:54:03,636 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/19ac8b6eb4cf42558eb6d0ea4bd99635 for inclusion in d05b1269482a465dff6cf06fb1c98132/family1 2023-03-22 15:54:03,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-22 15:54:03,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:03,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:54:03,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/19ac8b6eb4cf42558eb6d0ea4bd99635 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__jk43403gj6rr200kdopu2kjoamk8dnudbrfubu3np0v3a71aigoigkok45shi85b/family1/19ac8b6eb4cf42558eb6d0ea4bd99635 2023-03-22 15:54:03,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__jk43403gj6rr200kdopu2kjoamk8dnudbrfubu3np0v3a71aigoigkok45shi85b/family1/19ac8b6eb4cf42558eb6d0ea4bd99635 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b283f764e7c84ebaa30323ce6070e065_SeqId_44_ 2023-03-22 15:54:03,643 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__jk43403gj6rr200kdopu2kjoamk8dnudbrfubu3np0v3a71aigoigkok45shi85b/family1/19ac8b6eb4cf42558eb6d0ea4bd99635 into d05b1269482a465dff6cf06fb1c98132/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b283f764e7c84ebaa30323ce6070e065_SeqId_44_ - updating store file list. 2023-03-22 15:54:03,648 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b283f764e7c84ebaa30323ce6070e065_SeqId_44_ into d05b1269482a465dff6cf06fb1c98132/family1 2023-03-22 15:54:03,648 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__jk43403gj6rr200kdopu2kjoamk8dnudbrfubu3np0v3a71aigoigkok45shi85b/family1/19ac8b6eb4cf42558eb6d0ea4bd99635 into d05b1269482a465dff6cf06fb1c98132/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b283f764e7c84ebaa30323ce6070e065_SeqId_44_) 2023-03-22 15:54:03,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__jk43403gj6rr200kdopu2kjoamk8dnudbrfubu3np0v3a71aigoigkok45shi85b/family1/19ac8b6eb4cf42558eb6d0ea4bd99635 2023-03-22 15:54:03,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-22 15:54:03,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-22 15:54:03,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): d05b1269482a465dff6cf06fb1c98132/family1 is initiating minor compaction 2023-03-22 15:54:03,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1c083843; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:03,653 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d05b1269482a465dff6cf06fb1c98132/family1 in testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:54:03,653 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/7b48444d3a09404daf7358c9e2b6e952, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/2616a8adec1241308e9aaa7e14929c45, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/f6eeb6713c0549aa89c79b457674a121, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/5b0711e3cb3f4e5e9beef46a4bd70a48, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/14b7856f55964796834e8b6bbbe79250, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/e5754516a20e4a10bc5dcaff09713660, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/05c9fe509c4b45c0a4459ad81717661d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a22b36f8a8ae4523a9df45442f08fcdd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a9e47df789af41e399e3c7aaa55507c4, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b283f764e7c84ebaa30323ce6070e065_SeqId_44_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp, totalSize=49.7 K 2023-03-22 15:54:03,653 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 7b48444d3a09404daf7358c9e2b6e952, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-22 15:54:03,654 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 2616a8adec1241308e9aaa7e14929c45, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-22 15:54:03,654 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting f6eeb6713c0549aa89c79b457674a121, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-22 15:54:03,654 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 5b0711e3cb3f4e5e9beef46a4bd70a48, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-22 15:54:03,654 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 14b7856f55964796834e8b6bbbe79250, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-22 15:54:03,654 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting e5754516a20e4a10bc5dcaff09713660, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-22 15:54:03,654 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 05c9fe509c4b45c0a4459ad81717661d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-22 15:54:03,654 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting a22b36f8a8ae4523a9df45442f08fcdd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-22 15:54:03,654 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting a9e47df789af41e399e3c7aaa55507c4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-22 15:54:03,654 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting b283f764e7c84ebaa30323ce6070e065_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-22 15:54:03,656 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:54:03,657 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:03,659 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101e84a1a53270c4714b3e6d11cf8ab21df to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101e84a1a53270c4714b3e6d11cf8ab21df 2023-03-22 15:54:03,660 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601031a0b221455384b8aabe8f1814e6cad17 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601031a0b221455384b8aabe8f1814e6cad17 2023-03-22 15:54:03,660 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-22 15:54:03,660 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-22 15:54:03,661 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:54:03,661 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:54:03,721 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): d05b1269482a465dff6cf06fb1c98132#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:54:04,149 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231b4e69bf23fad4ee6ad9c85214bfaa8ff to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231b4e69bf23fad4ee6ad9c85214bfaa8ff 2023-03-22 15:54:04,150 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:54:04,176 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d4a0a1b47ba049f0b1326dfcf4914bab first=Optional[r1] last=Optional[r4] 2023-03-22 15:54:04,176 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d4a0a1b47ba049f0b1326dfcf4914bab}] 2023-03-22 15:54:04,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d4a0a1b47ba049f0b1326dfcf4914bab for inclusion in d05b1269482a465dff6cf06fb1c98132/family1 2023-03-22 15:54:04,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-22 15:54:04,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:04,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:54:04,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/d4a0a1b47ba049f0b1326dfcf4914bab to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__85n57249c4k0q8hsjshugcvu8otl0lto4i1evpthbjl3du3kfb8d6p722ir6i6mp/family1/d4a0a1b47ba049f0b1326dfcf4914bab 2023-03-22 15:54:04,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__85n57249c4k0q8hsjshugcvu8otl0lto4i1evpthbjl3du3kfb8d6p722ir6i6mp/family1/d4a0a1b47ba049f0b1326dfcf4914bab as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/77f1d7750268493ab4cb8e574e3732c1_SeqId_46_ 2023-03-22 15:54:04,186 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__85n57249c4k0q8hsjshugcvu8otl0lto4i1evpthbjl3du3kfb8d6p722ir6i6mp/family1/d4a0a1b47ba049f0b1326dfcf4914bab into d05b1269482a465dff6cf06fb1c98132/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/77f1d7750268493ab4cb8e574e3732c1_SeqId_46_ - updating store file list. 2023-03-22 15:54:04,191 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/77f1d7750268493ab4cb8e574e3732c1_SeqId_46_ into d05b1269482a465dff6cf06fb1c98132/family1 2023-03-22 15:54:04,191 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__85n57249c4k0q8hsjshugcvu8otl0lto4i1evpthbjl3du3kfb8d6p722ir6i6mp/family1/d4a0a1b47ba049f0b1326dfcf4914bab into d05b1269482a465dff6cf06fb1c98132/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/77f1d7750268493ab4cb8e574e3732c1_SeqId_46_) 2023-03-22 15:54:04,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__85n57249c4k0q8hsjshugcvu8otl0lto4i1evpthbjl3du3kfb8d6p722ir6i6mp/family1/d4a0a1b47ba049f0b1326dfcf4914bab 2023-03-22 15:54:04,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:54:04,194 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:04,196 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201452550d91a974c938bbbc9ac1767af55 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201452550d91a974c938bbbc9ac1767af55 2023-03-22 15:54:04,197 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512056f66d0682d454e43a3b3e226bba54778 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512056f66d0682d454e43a3b3e226bba54778 2023-03-22 15:54:04,198 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122890dfcc7329ba40c6baeec4dfa819f749 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122890dfcc7329ba40c6baeec4dfa819f749 2023-03-22 15:54:04,199 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123169ff0794ac44479599b858a8059b6d21 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123169ff0794ac44479599b858a8059b6d21 2023-03-22 15:54:04,199 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-22 15:54:04,199 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-22 15:54:04,200 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:54:04,200 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:54:04,548 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/bc63fc2e29614f93b249746e33c3ef0e as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/bc63fc2e29614f93b249746e33c3ef0e 2023-03-22 15:54:04,554 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d05b1269482a465dff6cf06fb1c98132/family1 of d05b1269482a465dff6cf06fb1c98132 into bc63fc2e29614f93b249746e33c3ef0e(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:04,554 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:54:04,554 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1679500443653; duration=0sec 2023-03-22 15:54:04,554 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:04,605 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/7b48444d3a09404daf7358c9e2b6e952, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/2616a8adec1241308e9aaa7e14929c45, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/f6eeb6713c0549aa89c79b457674a121, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/5b0711e3cb3f4e5e9beef46a4bd70a48, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/14b7856f55964796834e8b6bbbe79250, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/e5754516a20e4a10bc5dcaff09713660, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/05c9fe509c4b45c0a4459ad81717661d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a22b36f8a8ae4523a9df45442f08fcdd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a9e47df789af41e399e3c7aaa55507c4, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b283f764e7c84ebaa30323ce6070e065_SeqId_44_] to archive 2023-03-22 15:54:04,606 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:04,608 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/7b48444d3a09404daf7358c9e2b6e952 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/7b48444d3a09404daf7358c9e2b6e952 2023-03-22 15:54:04,610 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/2616a8adec1241308e9aaa7e14929c45 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/2616a8adec1241308e9aaa7e14929c45 2023-03-22 15:54:04,611 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/f6eeb6713c0549aa89c79b457674a121 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/f6eeb6713c0549aa89c79b457674a121 2023-03-22 15:54:04,612 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/5b0711e3cb3f4e5e9beef46a4bd70a48 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/5b0711e3cb3f4e5e9beef46a4bd70a48 2023-03-22 15:54:04,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/14b7856f55964796834e8b6bbbe79250 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/14b7856f55964796834e8b6bbbe79250 2023-03-22 15:54:04,615 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/e5754516a20e4a10bc5dcaff09713660 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/e5754516a20e4a10bc5dcaff09713660 2023-03-22 15:54:04,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/05c9fe509c4b45c0a4459ad81717661d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/05c9fe509c4b45c0a4459ad81717661d 2023-03-22 15:54:04,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a22b36f8a8ae4523a9df45442f08fcdd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a22b36f8a8ae4523a9df45442f08fcdd 2023-03-22 15:54:04,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a9e47df789af41e399e3c7aaa55507c4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/a9e47df789af41e399e3c7aaa55507c4 2023-03-22 15:54:04,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b283f764e7c84ebaa30323ce6070e065_SeqId_44_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b283f764e7c84ebaa30323ce6070e065_SeqId_44_ 2023-03-22 15:54:05,045 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130518ded8603c54618a9ee5579cee5c1d3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130518ded8603c54618a9ee5579cee5c1d3 2023-03-22 15:54:05,046 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:54:05,072 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b09199eaedbb48fcaeab37662f886fc2 first=Optional[r00] last=Optional[r8] 2023-03-22 15:54:05,073 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b09199eaedbb48fcaeab37662f886fc2}] 2023-03-22 15:54:05,075 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b09199eaedbb48fcaeab37662f886fc2 for inclusion in d05b1269482a465dff6cf06fb1c98132/family1 2023-03-22 15:54:05,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-22 15:54:05,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:05,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:54:05,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b09199eaedbb48fcaeab37662f886fc2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6p0m5496kipv2a2boolscf2qijbksbm3h1m998pvvr2kfbjh67p6m6k2aqlekeer/family1/b09199eaedbb48fcaeab37662f886fc2 2023-03-22 15:54:05,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6p0m5496kipv2a2boolscf2qijbksbm3h1m998pvvr2kfbjh67p6m6k2aqlekeer/family1/b09199eaedbb48fcaeab37662f886fc2 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b7f26d43035943dcbaf685ecc929f752_SeqId_49_ 2023-03-22 15:54:05,083 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6p0m5496kipv2a2boolscf2qijbksbm3h1m998pvvr2kfbjh67p6m6k2aqlekeer/family1/b09199eaedbb48fcaeab37662f886fc2 into d05b1269482a465dff6cf06fb1c98132/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b7f26d43035943dcbaf685ecc929f752_SeqId_49_ - updating store file list. 2023-03-22 15:54:05,089 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b7f26d43035943dcbaf685ecc929f752_SeqId_49_ into d05b1269482a465dff6cf06fb1c98132/family1 2023-03-22 15:54:05,089 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6p0m5496kipv2a2boolscf2qijbksbm3h1m998pvvr2kfbjh67p6m6k2aqlekeer/family1/b09199eaedbb48fcaeab37662f886fc2 into d05b1269482a465dff6cf06fb1c98132/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b7f26d43035943dcbaf685ecc929f752_SeqId_49_) 2023-03-22 15:54:05,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6p0m5496kipv2a2boolscf2qijbksbm3h1m998pvvr2kfbjh67p6m6k2aqlekeer/family1/b09199eaedbb48fcaeab37662f886fc2 2023-03-22 15:54:05,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-22 15:54:05,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-22 15:54:05,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): d05b1269482a465dff6cf06fb1c98132/family1 is initiating minor compaction (all files) 2023-03-22 15:54:05,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@28f3f354; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:05,091 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d05b1269482a465dff6cf06fb1c98132/family1 in testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:54:05,092 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/0bf968c846c34d6fa976fd5f699baa02, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/bc63fc2e29614f93b249746e33c3ef0e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/77f1d7750268493ab4cb8e574e3732c1_SeqId_46_, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b7f26d43035943dcbaf685ecc929f752_SeqId_49_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp, totalSize=344.3 K 2023-03-22 15:54:05,092 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 0bf968c846c34d6fa976fd5f699baa02, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840482 2023-03-22 15:54:05,093 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting bc63fc2e29614f93b249746e33c3ef0e, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240482 2023-03-22 15:54:05,093 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 77f1d7750268493ab4cb8e574e3732c1_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965240482 2023-03-22 15:54:05,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:54:05,094 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting b7f26d43035943dcbaf685ecc929f752_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014840482 2023-03-22 15:54:05,094 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:05,096 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120a9d6ec1ee826459fb7c4fbf52e75558a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120a9d6ec1ee826459fb7c4fbf52e75558a 2023-03-22 15:54:05,097 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112810dfee70739b4c918f343d2be5c1243a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112810dfee70739b4c918f343d2be5c1243a 2023-03-22 15:54:05,099 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511301288c1487edb4c479ed649feaa398d10 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511301288c1487edb4c479ed649feaa398d10 2023-03-22 15:54:05,099 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-22 15:54:05,099 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-22 15:54:05,099 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:54:05,099 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:54:05,099 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:54:05,099 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:54:05,100 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:05,101 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:54:05,102 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.3570 sec 2023-03-22 15:54:05,119 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): d05b1269482a465dff6cf06fb1c98132#family1#compaction#23 average throughput is 11.97 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:54:05,208 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=258 (was 248) Potentially hanging thread: hconnection-0x47645d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x47645d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2054959231_15 at /127.0.0.1:55736 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55800 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55906 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58225@0x1dbf9cfd sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1580181509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55810 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55820 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x47645d-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x47645d-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:36540 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x47645d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2054959231_15 at /127.0.0.1:55930 [Receiving block BP-1976223936-172.31.14.131-1679500411633:blk_1073741900_1076] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58225@0x1dbf9cfd-SendThread(127.0.0.1:58225) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55702 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:58788 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130935b6df700f241eeaf578ea2ff5a91aa java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:58784 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55788 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:58762 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:58225@0x1dbf9cfd-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x47645d-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55780 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55714 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:58794 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/478cf0c1acfa4cbcb7132af3fdccd201 block BP-1976223936-172.31.14.131-1679500411633:blk_1073741900_1076 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: PacketResponder: BP-1976223936-172.31.14.131-1679500411633:blk_1073741900_1076, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x47645d-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55916 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ResponseProcessor for block BP-1976223936-172.31.14.131-1679500411633:blk_1073741900_1076 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1606816495_15 at /127.0.0.1:55724 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=612 (was 599) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=445 (was 437) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=6865 (was 7003) 2023-03-22 15:54:05,220 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=258, OpenFileDescriptor=612, MaxFileDescriptor=60000, SystemLoadAverage=445, ProcessCount=172, AvailableMemoryMB=6864 2023-03-22 15:54:05,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-22 15:54:05,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:05,225 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:54:05,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-22 15:54:05,226 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:54:05,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-22 15:54:05,228 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262 2023-03-22 15:54:05,229 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262 empty. 2023-03-22 15:54:05,229 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262 2023-03-22 15:54:05,229 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-22 15:54:05,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-22 15:54:05,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-22 15:54:05,642 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-22 15:54:05,643 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => ec32694136837c24213a5ee256a95262, NAME => 'testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:05,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-22 15:54:05,948 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/.tmp/family1/478cf0c1acfa4cbcb7132af3fdccd201 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/478cf0c1acfa4cbcb7132af3fdccd201 2023-03-22 15:54:05,954 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d05b1269482a465dff6cf06fb1c98132/family1 of d05b1269482a465dff6cf06fb1c98132 into 478cf0c1acfa4cbcb7132af3fdccd201(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:05,955 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:54:05,955 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1679500445091; duration=0sec 2023-03-22 15:54:05,955 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:05,993 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/0bf968c846c34d6fa976fd5f699baa02, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/bc63fc2e29614f93b249746e33c3ef0e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/77f1d7750268493ab4cb8e574e3732c1_SeqId_46_, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b7f26d43035943dcbaf685ecc929f752_SeqId_49_] to archive 2023-03-22 15:54:05,995 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:05,997 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/0bf968c846c34d6fa976fd5f699baa02 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/0bf968c846c34d6fa976fd5f699baa02 2023-03-22 15:54:05,998 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/bc63fc2e29614f93b249746e33c3ef0e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/bc63fc2e29614f93b249746e33c3ef0e 2023-03-22 15:54:05,999 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/77f1d7750268493ab4cb8e574e3732c1_SeqId_46_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/77f1d7750268493ab4cb8e574e3732c1_SeqId_46_ 2023-03-22 15:54:06,001 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b7f26d43035943dcbaf685ecc929f752_SeqId_49_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/family1/b7f26d43035943dcbaf685ecc929f752_SeqId_49_ 2023-03-22 15:54:06,050 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:06,050 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing ec32694136837c24213a5ee256a95262, disabling compactions & flushes 2023-03-22 15:54:06,050 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:06,050 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:06,050 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. after waiting 0 ms 2023-03-22 15:54:06,050 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:06,050 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:06,050 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:06,053 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:54:06,054 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679500446053"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500446053"}]},"ts":"1679500446053"} 2023-03-22 15:54:06,055 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-22 15:54:06,056 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:54:06,056 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500446056"}]},"ts":"1679500446056"} 2023-03-22 15:54:06,058 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-22 15:54:06,064 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=ec32694136837c24213a5ee256a95262, ASSIGN}] 2023-03-22 15:54:06,065 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=ec32694136837c24213a5ee256a95262, ASSIGN 2023-03-22 15:54:06,066 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=ec32694136837c24213a5ee256a95262, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:06,217 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=ec32694136837c24213a5ee256a95262, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:06,217 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679500446217"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500446217"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500446217"}]},"ts":"1679500446217"} 2023-03-22 15:54:06,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure ec32694136837c24213a5ee256a95262, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:06,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-22 15:54:06,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ec32694136837c24213a5ee256a95262, NAME => 'testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:54:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy ec32694136837c24213a5ee256a95262 2023-03-22 15:54:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ec32694136837c24213a5ee256a95262 2023-03-22 15:54:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ec32694136837c24213a5ee256a95262 2023-03-22 15:54:06,377 INFO [StoreOpener-ec32694136837c24213a5ee256a95262-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ec32694136837c24213a5ee256a95262 2023-03-22 15:54:06,378 DEBUG [StoreOpener-ec32694136837c24213a5ee256a95262-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1 2023-03-22 15:54:06,378 DEBUG [StoreOpener-ec32694136837c24213a5ee256a95262-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1 2023-03-22 15:54:06,378 INFO [StoreOpener-ec32694136837c24213a5ee256a95262-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec32694136837c24213a5ee256a95262 columnFamilyName family1 2023-03-22 15:54:06,380 INFO [StoreOpener-ec32694136837c24213a5ee256a95262-1] regionserver.HStore(310): Store=ec32694136837c24213a5ee256a95262/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:06,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262 2023-03-22 15:54:06,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262 2023-03-22 15:54:06,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ec32694136837c24213a5ee256a95262 2023-03-22 15:54:06,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:06,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ec32694136837c24213a5ee256a95262; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60510637, jitterRate=-0.09832124412059784}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:54:06,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:06,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262., pid=25, masterSystemTime=1679500446371 2023-03-22 15:54:06,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:06,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:06,389 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=ec32694136837c24213a5ee256a95262, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:06,390 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679500446389"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500446389"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500446389"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500446389"}]},"ts":"1679500446389"} 2023-03-22 15:54:06,393 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-22 15:54:06,393 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure ec32694136837c24213a5ee256a95262, server=jenkins-hbase4.apache.org,38637,1679500416185 in 172 msec 2023-03-22 15:54:06,395 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-22 15:54:06,395 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=ec32694136837c24213a5ee256a95262, ASSIGN in 329 msec 2023-03-22 15:54:06,396 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:54:06,396 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500446396"}]},"ts":"1679500446396"} 2023-03-22 15:54:06,398 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-22 15:54:06,400 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:54:06,401 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1780 sec 2023-03-22 15:54:07,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-22 15:54:07,333 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-22 15:54:07,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:07,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:07,359 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:07,359 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:07,359 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:07,359 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:07,360 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:07,360 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:07,361 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:07,361 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:07,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:07,361 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:07,362 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:07,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:07,362 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:07,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:07,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:07,362 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:07,364 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:07,364 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:07,364 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:07,364 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:07,364 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:07,364 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:07,364 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:07,364 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:07,365 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-22 15:54:07,782 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130344d7c7bbc1f4ed6b2c1485044b37738 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130344d7c7bbc1f4ed6b2c1485044b37738 2023-03-22 15:54:07,958 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:08,186 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/bf23fe277f4d49cba5c5d29721d79184 2023-03-22 15:54:08,192 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/bf23fe277f4d49cba5c5d29721d79184 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/bf23fe277f4d49cba5c5d29721d79184 2023-03-22 15:54:08,196 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/bf23fe277f4d49cba5c5d29721d79184, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-22 15:54:08,198 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 832ms, sequenceid=5, compaction requested=false 2023-03-22 15:54:08,198 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:08,198 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:08,198 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:08,198 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:08,198 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,198 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:08,199 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,200 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:08,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:08,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:08,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:08,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:08,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f670738[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:08,202 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:08,202 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,203 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:08,203 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:08,203 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:08,205 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:08,205 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,205 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:08,205 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:08,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:08,208 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:08,208 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:08,208 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:08,208 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,208 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:08,210 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,210 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:08,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,210 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:08,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:08,211 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,212 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,214 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,214 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:08,214 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:08,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:08,214 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,214 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:08,214 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:08,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:08,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:08,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:08,214 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,215 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,215 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,215 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:08,215 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:08,215 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:08,315 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:08,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:08,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:08,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:08,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,323 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:08,323 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:08,323 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:08,323 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:08,323 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,323 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,325 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:08,325 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:08,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:08,325 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:08,325 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:08,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:08,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,326 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:08,326 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:08,326 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:08,326 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:08,326 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:08,326 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:08,326 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:08,326 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:08,327 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:08,327 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-22 15:54:08,742 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c587486bbdac4ad0998bd1c75363aa22 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128c587486bbdac4ad0998bd1c75363aa22 2023-03-22 15:54:09,148 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/756fd9a22a70445ca0dbe4dbce706bfe 2023-03-22 15:54:09,154 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/756fd9a22a70445ca0dbe4dbce706bfe as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/756fd9a22a70445ca0dbe4dbce706bfe 2023-03-22 15:54:09,158 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/756fd9a22a70445ca0dbe4dbce706bfe, entries=1, sequenceid=9, filesize=5.0 K 2023-03-22 15:54:09,159 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 832ms, sequenceid=9, compaction requested=false 2023-03-22 15:54:09,160 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:09,160 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:09,160 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:09,160 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:09,160 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,160 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:09,160 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:09,162 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,162 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:09,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:09,162 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,162 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:09,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:09,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:09,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:09,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:09,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56ab5ace[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:09,164 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:09,164 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,166 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,166 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:09,166 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:09,166 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:09,166 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,166 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:09,167 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:09,167 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,167 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:09,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:09,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:09,167 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:09,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:09,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:09,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:09,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:09,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:09,170 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:09,170 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:09,170 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:09,170 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,170 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:09,172 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,172 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,172 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:09,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,172 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:09,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,173 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:09,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,175 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,175 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,175 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:09,175 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:09,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:09,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:09,176 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:09,176 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,175 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:09,176 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,176 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,176 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:09,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:09,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:09,176 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:09,176 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:09,276 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:09,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:09,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:09,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:09,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,283 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:09,284 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:09,284 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:09,284 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:09,284 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,284 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,286 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:09,286 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:09,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:09,287 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:09,287 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:09,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:09,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,288 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:09,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:09,288 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:09,288 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:09,288 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:09,288 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:09,288 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:09,288 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:09,288 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:09,288 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:09,702 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201ce9f299d7f004986831a2b336ef1efa8 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201ce9f299d7f004986831a2b336ef1efa8 2023-03-22 15:54:10,108 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/364749f34f584025973379102c5ffa6c 2023-03-22 15:54:10,114 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/364749f34f584025973379102c5ffa6c as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/364749f34f584025973379102c5ffa6c 2023-03-22 15:54:10,119 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/364749f34f584025973379102c5ffa6c, entries=1, sequenceid=13, filesize=4.9 K 2023-03-22 15:54:10,120 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 832ms, sequenceid=13, compaction requested=true 2023-03-22 15:54:10,120 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:10,120 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:10,120 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:10,120 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:10,120 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,121 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:10,121 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:10,122 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,122 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:10,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:10,123 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:10,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:10,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61b364da[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:10,124 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:10,124 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,126 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,126 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:10,126 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:10,126 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,126 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:10,126 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:10,128 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,128 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:10,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:10,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:10,128 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:10,128 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:10,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:10,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:10,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:10,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:10,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:10,135 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:10,135 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:10,135 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:10,135 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,135 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:10,138 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,138 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,138 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:10,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:10,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:10,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:10,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:10,138 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:10,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,139 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:10,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,144 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,144 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,144 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,144 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:10,144 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:10,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:10,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:10,144 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:10,145 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:10,145 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,145 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,145 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:10,145 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:10,145 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:10,246 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:10,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:10,251 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:10,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:10,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,253 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:10,253 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:10,253 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:10,253 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:10,254 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,254 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,255 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,255 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:10,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:10,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:10,255 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:10,256 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,256 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:10,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,257 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:10,257 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:10,257 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:10,257 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:10,257 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:10,257 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:10,257 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:10,257 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:10,257 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:10,257 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:10,672 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120597a4d90b954441639d892647bbb88b72 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120597a4d90b954441639d892647bbb88b72 2023-03-22 15:54:11,078 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/94c5314a330f43b5bb9e183d945ced39 2023-03-22 15:54:11,083 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/94c5314a330f43b5bb9e183d945ced39 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/94c5314a330f43b5bb9e183d945ced39 2023-03-22 15:54:11,088 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/94c5314a330f43b5bb9e183d945ced39, entries=1, sequenceid=17, filesize=4.9 K 2023-03-22 15:54:11,089 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 832ms, sequenceid=17, compaction requested=true 2023-03-22 15:54:11,089 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:11,089 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:11,089 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:11,089 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:11,090 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,090 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:11,090 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:11,091 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,091 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:11,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:11,091 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,092 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:11,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:11,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:11,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:11,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:11,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a06f440[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:11,093 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:11,093 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,095 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,095 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:11,095 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:11,095 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,095 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:11,095 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:11,096 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,096 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:11,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:11,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:11,096 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:11,097 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:11,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:11,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:11,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:11,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:11,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:11,098 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:11,099 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:11,099 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:11,099 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,099 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:11,100 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,100 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:11,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:11,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,100 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:11,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:11,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:11,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:11,101 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:11,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,102 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:11,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:11,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:11,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:11,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:11,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:11,104 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:11,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:11,105 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,105 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:11,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:11,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:11,105 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,105 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:11,105 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,105 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,205 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:11,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:11,210 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:11,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:11,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,212 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:11,212 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:11,212 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:11,212 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:11,213 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,213 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,217 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,217 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:11,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:11,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:11,218 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:11,218 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:11,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,218 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:11,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,218 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,219 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:11,219 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:11,219 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:11,219 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:11,219 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:11,219 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:11,219 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:11,219 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:11,219 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:11,219 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:11,634 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d593fcd08f4b481d9cf430455cebcd26 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d593fcd08f4b481d9cf430455cebcd26 2023-03-22 15:54:11,909 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:54:11,949 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:54:11,950 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1dc63c7f to 127.0.0.1:58225 2023-03-22 15:54:11,950 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:54:11,951 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:54:11,951 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1dbf9cfd to 127.0.0.1:58225 2023-03-22 15:54:11,951 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:54:11,951 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:54:11,951 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x621d1842 to 127.0.0.1:58225 2023-03-22 15:54:11,951 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:54:12,040 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/753191d77cda4f6280f62005683aabb3 2023-03-22 15:54:12,045 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/753191d77cda4f6280f62005683aabb3 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/753191d77cda4f6280f62005683aabb3 2023-03-22 15:54:12,049 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/753191d77cda4f6280f62005683aabb3, entries=1, sequenceid=21, filesize=4.9 K 2023-03-22 15:54:12,050 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 831ms, sequenceid=21, compaction requested=true 2023-03-22 15:54:12,050 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:12,051 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:12,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:12,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:12,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:12,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:12,054 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,054 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:12,054 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:12,054 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:12,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:12,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:12,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d8f7eff[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:12,056 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:12,056 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,057 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,057 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:12,057 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:12,057 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:12,057 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,057 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:12,059 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:12,059 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,059 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:12,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,059 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:12,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:12,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:12,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:12,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:12,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:12,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:12,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:12,061 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:12,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:12,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:12,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,061 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:12,063 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,063 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,063 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:12,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:12,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:12,063 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:12,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:12,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:12,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:12,064 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:12,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,071 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,071 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:12,071 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:12,071 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,071 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:12,071 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:12,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:12,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:12,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:12,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:12,071 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,072 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,072 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,072 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:12,072 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:12,072 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:12,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:12,172 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:12,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:12,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:12,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:12,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,179 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:12,180 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:12,180 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:12,180 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:12,180 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,180 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,182 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,182 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:12,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:12,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:12,182 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:12,182 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:12,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:12,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:12,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:12,183 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:12,183 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:12,183 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:12,183 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:12,183 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:12,183 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:12,183 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:12,184 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:12,596 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231f56e1f868ff14f689aa92fb663e8b97c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231f56e1f868ff14f689aa92fb663e8b97c 2023-03-22 15:54:13,003 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/48296c99255948779467afff74642e22 2023-03-22 15:54:13,019 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/48296c99255948779467afff74642e22 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/48296c99255948779467afff74642e22 2023-03-22 15:54:13,040 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/48296c99255948779467afff74642e22, entries=1, sequenceid=25, filesize=4.9 K 2023-03-22 15:54:13,052 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 869ms, sequenceid=25, compaction requested=true 2023-03-22 15:54:13,052 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:13,052 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:13,052 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:13,052 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:13,052 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,053 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:13,053 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:13,055 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,055 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:13,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:13,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:13,055 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,056 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:13,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:13,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:13,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:13,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41fe20aa[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:13,059 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:13,059 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,061 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,062 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:13,062 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:13,062 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:13,062 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,062 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:13,068 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,068 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:13,069 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:13,069 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:13,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:13,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:13,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:13,072 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:13,072 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:13,072 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:13,073 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,073 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:13,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:13,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:13,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:13,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:13,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:13,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:13,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,076 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:13,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,077 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:13,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,082 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:13,082 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:13,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:13,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:13,083 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:13,083 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:13,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:13,083 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:13,083 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,083 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,083 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,083 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,083 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,083 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:13,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:13,084 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:13,084 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:13,184 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:13,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:13,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:13,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:13,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,208 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:13,208 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:13,208 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:13,208 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:13,210 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,210 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,213 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:13,213 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:13,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:13,213 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,213 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:13,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:13,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:13,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,228 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:13,228 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:13,228 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:13,236 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:13,236 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:13,236 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:13,236 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:13,237 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:13,708 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101122c0aae21454a6894c3e923a3f87431 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101122c0aae21454a6894c3e923a3f87431 2023-03-22 15:54:13,715 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/718986fe127b4c97b403f34c04a0d5ea 2023-03-22 15:54:13,724 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/718986fe127b4c97b403f34c04a0d5ea as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/718986fe127b4c97b403f34c04a0d5ea 2023-03-22 15:54:13,730 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/718986fe127b4c97b403f34c04a0d5ea, entries=1, sequenceid=29, filesize=4.9 K 2023-03-22 15:54:13,731 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 494ms, sequenceid=29, compaction requested=true 2023-03-22 15:54:13,731 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:13,731 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:13,731 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:13,733 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,733 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:13,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:13,734 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,734 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:13,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:13,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:13,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:13,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:13,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b1109fb[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:13,735 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:13,735 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,738 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,738 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:13,738 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,738 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:13,738 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:13,738 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:13,739 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:13,739 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,739 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:13,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:13,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:13,739 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:13,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:13,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:13,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:13,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:13,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:13,742 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:13,742 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:13,742 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:13,742 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,742 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:13,744 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,744 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:13,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,744 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:13,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:13,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:13,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:13,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:13,744 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:13,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,746 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:13,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,748 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,748 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:13,748 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,748 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:13,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:13,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:13,748 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:13,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:13,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:13,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:13,748 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,749 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,749 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,749 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,749 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:13,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:13,749 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:13,749 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:13,849 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:13,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:13,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:13,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:13,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,857 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:13,858 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:13,858 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:13,858 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:13,858 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,858 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,861 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:13,861 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:13,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:13,861 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,861 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:13,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:13,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,863 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:13,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,864 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:13,864 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:13,864 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:13,864 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:13,864 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:13,864 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:13,864 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:13,864 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:13,880 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103eca393955d4f43938a17ab67a1682f13 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103eca393955d4f43938a17ab67a1682f13 2023-03-22 15:54:13,902 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/2bdc63dccf88428a819ec30abf9c9392 2023-03-22 15:54:13,911 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/2bdc63dccf88428a819ec30abf9c9392 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/2bdc63dccf88428a819ec30abf9c9392 2023-03-22 15:54:13,917 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/2bdc63dccf88428a819ec30abf9c9392, entries=1, sequenceid=33, filesize=4.9 K 2023-03-22 15:54:13,918 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 54ms, sequenceid=33, compaction requested=true 2023-03-22 15:54:13,918 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:13,918 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:13,919 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:13,919 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:13,919 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,919 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:13,919 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:13,920 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,920 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:13,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:13,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:13,921 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,921 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:13,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:13,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:13,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:13,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16185e2d[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:13,923 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:13,923 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,924 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,925 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:13,925 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:13,925 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,925 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:13,925 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:13,927 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:13,927 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:13,927 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:13,927 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:13,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:13,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:13,932 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:13,932 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:13,932 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:13,932 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,932 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:13,936 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,936 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:13,936 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:13,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:13,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:13,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:13,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,939 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:13,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:13,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,942 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:13,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,945 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:13,945 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:13,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:13,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:13,945 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,945 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,945 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,945 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:13,945 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:13,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:13,945 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:13,945 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,946 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:13,946 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:13,946 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:13,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:13,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:13,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:14,046 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:14,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:14,054 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:14,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:14,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,061 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:14,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:14,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:14,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:14,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,063 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:14,063 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:14,063 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,063 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:14,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,064 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:14,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,064 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,066 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:14,066 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:14,066 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:14,079 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:14,079 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:14,079 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:14,111 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090792e28649e0294f478813fb251b1bad78 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015090792e28649e0294f478813fb251b1bad78 2023-03-22 15:54:14,531 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/679e4e28b2ac4e9bab1f60b8e23491c4 2023-03-22 15:54:14,538 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/679e4e28b2ac4e9bab1f60b8e23491c4 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/679e4e28b2ac4e9bab1f60b8e23491c4 2023-03-22 15:54:14,550 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/679e4e28b2ac4e9bab1f60b8e23491c4, entries=1, sequenceid=37, filesize=4.9 K 2023-03-22 15:54:14,551 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 472ms, sequenceid=37, compaction requested=true 2023-03-22 15:54:14,552 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:14,552 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:14,552 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:14,552 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:14,552 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,552 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:14,552 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:14,554 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,554 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:14,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:14,554 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,554 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:14,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:14,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:14,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:14,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:14,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2056b297[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:14,557 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:14,557 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,562 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:14,562 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:14,562 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:14,562 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,562 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:14,564 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:14,564 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,564 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:14,564 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:14,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:14,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:14,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:14,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:14,567 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:14,568 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:14,568 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:14,568 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,568 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:14,571 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,571 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:14,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:14,571 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:14,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:14,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:14,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:14,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,577 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:14,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:14,578 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,580 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,580 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:14,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:14,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:14,580 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,581 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,581 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:14,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:14,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:14,581 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:14,581 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:14,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:14,581 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,581 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:14,581 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:14,581 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,582 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,682 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:14,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:14,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:14,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:14,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,691 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:14,691 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:14,691 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:14,691 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:14,691 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,692 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,694 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:14,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:14,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:14,694 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,695 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:14,695 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:14,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:14,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:14,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:14,696 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:14,696 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:14,696 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:14,700 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:14,700 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:14,700 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:14,700 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. started... 2023-03-22 15:54:14,700 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ec32694136837c24213a5ee256a95262 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:15,143 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ff01aea25eb44d47b755fbd66cbe6fae to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120ff01aea25eb44d47b755fbd66cbe6fae 2023-03-22 15:54:15,151 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/648b42f280054f2f98c3d84c3ce0a078 2023-03-22 15:54:15,160 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/648b42f280054f2f98c3d84c3ce0a078 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/648b42f280054f2f98c3d84c3ce0a078 2023-03-22 15:54:15,169 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/648b42f280054f2f98c3d84c3ce0a078, entries=1, sequenceid=41, filesize=4.9 K 2023-03-22 15:54:15,170 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ec32694136837c24213a5ee256a95262 in 470ms, sequenceid=41, compaction requested=true 2023-03-22 15:54:15,170 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:15,170 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:15,170 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:15,171 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:15,172 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,173 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:15,173 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,173 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:15,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:15,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:15,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@101d2d47[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:15,175 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:15,175 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,177 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,177 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,177 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:15,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:15,177 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:15,179 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:15,179 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:15,179 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:15,179 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:15,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:15,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:15,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:15,183 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:15,183 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:15,183 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:15,183 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:15,183 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,183 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:15,186 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,186 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,186 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:15,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:15,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:15,187 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:15,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:15,189 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:15,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,192 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,192 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,192 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,192 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:15,192 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,192 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:15,192 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:15,193 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:15,193 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:15,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-22 15:54:15,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:15,192 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,193 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:15,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:15,195 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:15,195 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:15,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:15,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:15,295 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:15,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:15,297 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-22 15:54:15,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:54:15,300 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:54:15,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:15,301 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501055301 2023-03-22 15:54:15,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-22 15:54:15,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-22 15:54:15,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:15,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x63c134eb to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:15,316 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3fead680, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:15,316 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:15,822 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010303f3a3eaa0014e928cf51db61908f837 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010303f3a3eaa0014e928cf51db61908f837 2023-03-22 15:54:15,824 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:15,842 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:54:15,844 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42216, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:54:15,852 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:54:15,854 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60048, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:54:15,865 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f7dfdf9947444db0a06d77bf4cb142b0 first=Optional[r3] last=Optional[r6] 2023-03-22 15:54:15,866 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f7dfdf9947444db0a06d77bf4cb142b0}] 2023-03-22 15:54:15,872 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f7dfdf9947444db0a06d77bf4cb142b0 for inclusion in ec32694136837c24213a5ee256a95262/family1 2023-03-22 15:54:15,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-22 15:54:15,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:15,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:15,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f7dfdf9947444db0a06d77bf4cb142b0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cumu8gh56e6rst4q4p2is12p1o6dufnl4ao0sr7eek0oo8j25qe3sk5tlo90r4dh/family1/f7dfdf9947444db0a06d77bf4cb142b0 2023-03-22 15:54:15,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cumu8gh56e6rst4q4p2is12p1o6dufnl4ao0sr7eek0oo8j25qe3sk5tlo90r4dh/family1/f7dfdf9947444db0a06d77bf4cb142b0 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/56042c37796c42c38925c98badd51da7_SeqId_44_ 2023-03-22 15:54:15,884 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cumu8gh56e6rst4q4p2is12p1o6dufnl4ao0sr7eek0oo8j25qe3sk5tlo90r4dh/family1/f7dfdf9947444db0a06d77bf4cb142b0 into ec32694136837c24213a5ee256a95262/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/56042c37796c42c38925c98badd51da7_SeqId_44_ - updating store file list. 2023-03-22 15:54:15,898 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/56042c37796c42c38925c98badd51da7_SeqId_44_ into ec32694136837c24213a5ee256a95262/family1 2023-03-22 15:54:15,898 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cumu8gh56e6rst4q4p2is12p1o6dufnl4ao0sr7eek0oo8j25qe3sk5tlo90r4dh/family1/f7dfdf9947444db0a06d77bf4cb142b0 into ec32694136837c24213a5ee256a95262/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/56042c37796c42c38925c98badd51da7_SeqId_44_) 2023-03-22 15:54:15,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cumu8gh56e6rst4q4p2is12p1o6dufnl4ao0sr7eek0oo8j25qe3sk5tlo90r4dh/family1/f7dfdf9947444db0a06d77bf4cb142b0 2023-03-22 15:54:15,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-22 15:54:15,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-22 15:54:15,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): ec32694136837c24213a5ee256a95262/family1 is initiating minor compaction 2023-03-22 15:54:15,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@49aa0d4e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:15,905 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ec32694136837c24213a5ee256a95262/family1 in testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:54:15,905 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/756fd9a22a70445ca0dbe4dbce706bfe, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/364749f34f584025973379102c5ffa6c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/94c5314a330f43b5bb9e183d945ced39, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/753191d77cda4f6280f62005683aabb3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/48296c99255948779467afff74642e22, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/718986fe127b4c97b403f34c04a0d5ea, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/2bdc63dccf88428a819ec30abf9c9392, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/679e4e28b2ac4e9bab1f60b8e23491c4, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/648b42f280054f2f98c3d84c3ce0a078, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/56042c37796c42c38925c98badd51da7_SeqId_44_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp, totalSize=50.0 K 2023-03-22 15:54:15,905 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 756fd9a22a70445ca0dbe4dbce706bfe, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-22 15:54:15,905 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 364749f34f584025973379102c5ffa6c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-22 15:54:15,905 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 94c5314a330f43b5bb9e183d945ced39, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-22 15:54:15,905 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 753191d77cda4f6280f62005683aabb3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-22 15:54:15,906 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 48296c99255948779467afff74642e22, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-22 15:54:15,906 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 718986fe127b4c97b403f34c04a0d5ea, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-22 15:54:15,906 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 2bdc63dccf88428a819ec30abf9c9392, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-22 15:54:15,906 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 679e4e28b2ac4e9bab1f60b8e23491c4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-22 15:54:15,906 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 648b42f280054f2f98c3d84c3ce0a078, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-22 15:54:15,906 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 56042c37796c42c38925c98badd51da7_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-22 15:54:15,908 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:15,909 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:15,913 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d593fcd08f4b481d9cf430455cebcd26 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d593fcd08f4b481d9cf430455cebcd26 2023-03-22 15:54:15,921 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231f56e1f868ff14f689aa92fb663e8b97c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231f56e1f868ff14f689aa92fb663e8b97c 2023-03-22 15:54:15,923 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101122c0aae21454a6894c3e923a3f87431 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101122c0aae21454a6894c3e923a3f87431 2023-03-22 15:54:15,925 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103eca393955d4f43938a17ab67a1682f13 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103eca393955d4f43938a17ab67a1682f13 2023-03-22 15:54:15,925 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-22 15:54:15,925 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-22 15:54:15,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:15,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:15,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:15,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:15,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:15,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:15,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:15,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:16,008 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): ec32694136837c24213a5ee256a95262#family1#compaction#34 average throughput is 0.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:54:16,048 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120595877d43675e447ca67fa2191608efb5 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120595877d43675e447ca67fa2191608efb5 2023-03-22 15:54:16,049 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:16,076 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/272460fe7517451d83c8916da5b2ecba first=Optional[r1] last=Optional[r2] 2023-03-22 15:54:16,076 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/272460fe7517451d83c8916da5b2ecba}] 2023-03-22 15:54:16,080 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/272460fe7517451d83c8916da5b2ecba for inclusion in ec32694136837c24213a5ee256a95262/family1 2023-03-22 15:54:16,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-22 15:54:16,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:16,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:16,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/272460fe7517451d83c8916da5b2ecba to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__c224nkuks7sj18q54d0p9ng9q96s4kph1p89dum8aia7av6giu9k0av8ekssq1k6/family1/272460fe7517451d83c8916da5b2ecba 2023-03-22 15:54:16,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__c224nkuks7sj18q54d0p9ng9q96s4kph1p89dum8aia7av6giu9k0av8ekssq1k6/family1/272460fe7517451d83c8916da5b2ecba as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/3b2ee7b7e5c747988e645ed32cdf8d5c_SeqId_46_ 2023-03-22 15:54:16,089 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__c224nkuks7sj18q54d0p9ng9q96s4kph1p89dum8aia7av6giu9k0av8ekssq1k6/family1/272460fe7517451d83c8916da5b2ecba into ec32694136837c24213a5ee256a95262/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/3b2ee7b7e5c747988e645ed32cdf8d5c_SeqId_46_ - updating store file list. 2023-03-22 15:54:16,094 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/3b2ee7b7e5c747988e645ed32cdf8d5c_SeqId_46_ into ec32694136837c24213a5ee256a95262/family1 2023-03-22 15:54:16,094 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__c224nkuks7sj18q54d0p9ng9q96s4kph1p89dum8aia7av6giu9k0av8ekssq1k6/family1/272460fe7517451d83c8916da5b2ecba into ec32694136837c24213a5ee256a95262/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/3b2ee7b7e5c747988e645ed32cdf8d5c_SeqId_46_) 2023-03-22 15:54:16,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithWeeklyPolicy__c224nkuks7sj18q54d0p9ng9q96s4kph1p89dum8aia7av6giu9k0av8ekssq1k6/family1/272460fe7517451d83c8916da5b2ecba 2023-03-22 15:54:16,097 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:16,099 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:16,101 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201ce9f299d7f004986831a2b336ef1efa8 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201ce9f299d7f004986831a2b336ef1efa8 2023-03-22 15:54:16,102 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120597a4d90b954441639d892647bbb88b72 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120597a4d90b954441639d892647bbb88b72 2023-03-22 15:54:16,102 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-22 15:54:16,102 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-22 15:54:16,103 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:16,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:16,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:16,104 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:54:16,106 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 806 msec 2023-03-22 15:54:16,259 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=257 (was 258), OpenFileDescriptor=616 (was 612) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=433 (was 445), ProcessCount=170 (was 172), AvailableMemoryMB=6709 (was 6864) 2023-03-22 15:54:16,276 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=257, OpenFileDescriptor=616, MaxFileDescriptor=60000, SystemLoadAverage=433, ProcessCount=170, AvailableMemoryMB=6708 2023-03-22 15:54:16,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-22 15:54:16,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:16,282 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:54:16,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-22 15:54:16,283 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:54:16,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-22 15:54:16,288 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:16,288 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:16,292 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819 empty. 2023-03-22 15:54:16,292 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:16,292 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404 empty. 2023-03-22 15:54:16,293 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:16,293 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76 empty. 2023-03-22 15:54:16,293 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:16,293 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:16,293 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-22 15:54:16,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-22 15:54:16,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-22 15:54:16,709 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-22 15:54:16,711 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => 9183debe8c11b7195a71e3968c944c76, NAME => 'testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:16,711 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => c20178396b93c0ac2f980d2114beb404, NAME => 'testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:16,711 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4cb1d1539cd4a823e84ba5c732f22819, NAME => 'testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:16,740 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:16,740 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 4cb1d1539cd4a823e84ba5c732f22819, disabling compactions & flushes 2023-03-22 15:54:16,741 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:16,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:16,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. after waiting 0 ms 2023-03-22 15:54:16,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:16,741 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:16,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 4cb1d1539cd4a823e84ba5c732f22819: 2023-03-22 15:54:16,759 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:16,760 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing 9183debe8c11b7195a71e3968c944c76, disabling compactions & flushes 2023-03-22 15:54:16,760 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:16,760 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:16,760 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. after waiting 0 ms 2023-03-22 15:54:16,760 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:16,760 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:16,760 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for 9183debe8c11b7195a71e3968c944c76: 2023-03-22 15:54:16,868 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/.tmp/family1/b928b122551946209523b3f7ba4ac330 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/b928b122551946209523b3f7ba4ac330 2023-03-22 15:54:16,874 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in ec32694136837c24213a5ee256a95262/family1 of ec32694136837c24213a5ee256a95262 into b928b122551946209523b3f7ba4ac330(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:16,874 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:54:16,874 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1679500455904; duration=0sec 2023-03-22 15:54:16,875 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:16,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-22 15:54:16,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/756fd9a22a70445ca0dbe4dbce706bfe, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/364749f34f584025973379102c5ffa6c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/94c5314a330f43b5bb9e183d945ced39, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/753191d77cda4f6280f62005683aabb3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/48296c99255948779467afff74642e22, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/718986fe127b4c97b403f34c04a0d5ea, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/2bdc63dccf88428a819ec30abf9c9392, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/679e4e28b2ac4e9bab1f60b8e23491c4, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/648b42f280054f2f98c3d84c3ce0a078, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/56042c37796c42c38925c98badd51da7_SeqId_44_] to archive 2023-03-22 15:54:16,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:16,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/756fd9a22a70445ca0dbe4dbce706bfe to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/756fd9a22a70445ca0dbe4dbce706bfe 2023-03-22 15:54:16,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/364749f34f584025973379102c5ffa6c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/364749f34f584025973379102c5ffa6c 2023-03-22 15:54:16,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/94c5314a330f43b5bb9e183d945ced39 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/94c5314a330f43b5bb9e183d945ced39 2023-03-22 15:54:16,916 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/753191d77cda4f6280f62005683aabb3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/753191d77cda4f6280f62005683aabb3 2023-03-22 15:54:16,918 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/48296c99255948779467afff74642e22 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/48296c99255948779467afff74642e22 2023-03-22 15:54:16,919 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/718986fe127b4c97b403f34c04a0d5ea to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/718986fe127b4c97b403f34c04a0d5ea 2023-03-22 15:54:16,921 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/2bdc63dccf88428a819ec30abf9c9392 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/2bdc63dccf88428a819ec30abf9c9392 2023-03-22 15:54:16,923 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/679e4e28b2ac4e9bab1f60b8e23491c4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/679e4e28b2ac4e9bab1f60b8e23491c4 2023-03-22 15:54:16,925 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/648b42f280054f2f98c3d84c3ce0a078 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/648b42f280054f2f98c3d84c3ce0a078 2023-03-22 15:54:16,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/56042c37796c42c38925c98badd51da7_SeqId_44_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/family1/56042c37796c42c38925c98badd51da7_SeqId_44_ 2023-03-22 15:54:17,149 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:17,149 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing c20178396b93c0ac2f980d2114beb404, disabling compactions & flushes 2023-03-22 15:54:17,149 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:17,149 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:17,149 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. after waiting 0 ms 2023-03-22 15:54:17,149 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:17,149 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:17,149 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for c20178396b93c0ac2f980d2114beb404: 2023-03-22 15:54:17,153 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:54:17,154 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679500457154"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500457154"}]},"ts":"1679500457154"} 2023-03-22 15:54:17,154 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679500457154"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500457154"}]},"ts":"1679500457154"} 2023-03-22 15:54:17,154 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679500457154"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500457154"}]},"ts":"1679500457154"} 2023-03-22 15:54:17,160 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-22 15:54:17,161 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:54:17,161 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500457161"}]},"ts":"1679500457161"} 2023-03-22 15:54:17,163 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-22 15:54:17,169 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4cb1d1539cd4a823e84ba5c732f22819, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c20178396b93c0ac2f980d2114beb404, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=9183debe8c11b7195a71e3968c944c76, ASSIGN}] 2023-03-22 15:54:17,177 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=9183debe8c11b7195a71e3968c944c76, ASSIGN 2023-03-22 15:54:17,178 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c20178396b93c0ac2f980d2114beb404, ASSIGN 2023-03-22 15:54:17,178 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4cb1d1539cd4a823e84ba5c732f22819, ASSIGN 2023-03-22 15:54:17,179 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=9183debe8c11b7195a71e3968c944c76, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:17,179 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c20178396b93c0ac2f980d2114beb404, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:17,179 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4cb1d1539cd4a823e84ba5c732f22819, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:17,332 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=4cb1d1539cd4a823e84ba5c732f22819, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:17,332 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=9183debe8c11b7195a71e3968c944c76, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:17,332 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=c20178396b93c0ac2f980d2114beb404, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:17,332 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679500457332"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500457332"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500457332"}]},"ts":"1679500457332"} 2023-03-22 15:54:17,332 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679500457332"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500457332"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500457332"}]},"ts":"1679500457332"} 2023-03-22 15:54:17,332 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679500457332"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500457332"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500457332"}]},"ts":"1679500457332"} 2023-03-22 15:54:17,335 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure 4cb1d1539cd4a823e84ba5c732f22819, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:17,337 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure 9183debe8c11b7195a71e3968c944c76, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:17,338 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure c20178396b93c0ac2f980d2114beb404, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:17,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-22 15:54:17,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:17,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c20178396b93c0ac2f980d2114beb404, NAME => 'testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404.', STARTKEY => '1', ENDKEY => '2'} 2023-03-22 15:54:17,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:17,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:17,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:17,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:17,503 INFO [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:17,504 DEBUG [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404/family1 2023-03-22 15:54:17,504 DEBUG [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404/family1 2023-03-22 15:54:17,505 INFO [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c20178396b93c0ac2f980d2114beb404 columnFamilyName family1 2023-03-22 15:54:17,511 INFO [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] regionserver.HStore(310): Store=c20178396b93c0ac2f980d2114beb404/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:17,511 INFO [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:17,512 DEBUG [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404/family2 2023-03-22 15:54:17,512 DEBUG [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404/family2 2023-03-22 15:54:17,513 INFO [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c20178396b93c0ac2f980d2114beb404 columnFamilyName family2 2023-03-22 15:54:17,515 INFO [StoreOpener-c20178396b93c0ac2f980d2114beb404-1] regionserver.HStore(310): Store=c20178396b93c0ac2f980d2114beb404/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:17,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:17,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:17,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:54:17,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c20178396b93c0ac2f980d2114beb404 2023-03-22 15:54:17,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:17,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c20178396b93c0ac2f980d2114beb404; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75062513, jitterRate=0.11851860582828522}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:54:17,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c20178396b93c0ac2f980d2114beb404: 2023-03-22 15:54:17,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404., pid=33, masterSystemTime=1679500457489 2023-03-22 15:54:17,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:17,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:17,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:17,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4cb1d1539cd4a823e84ba5c732f22819, NAME => 'testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819.', STARTKEY => '', ENDKEY => '1'} 2023-03-22 15:54:17,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:17,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:17,527 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=c20178396b93c0ac2f980d2114beb404, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:17,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:17,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:17,527 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679500457527"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500457527"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500457527"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500457527"}]},"ts":"1679500457527"} 2023-03-22 15:54:17,531 INFO [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:17,532 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-03-22 15:54:17,532 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure c20178396b93c0ac2f980d2114beb404, server=jenkins-hbase4.apache.org,38637,1679500416185 in 191 msec 2023-03-22 15:54:17,534 DEBUG [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819/family1 2023-03-22 15:54:17,534 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c20178396b93c0ac2f980d2114beb404, ASSIGN in 363 msec 2023-03-22 15:54:17,534 DEBUG [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819/family1 2023-03-22 15:54:17,534 INFO [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4cb1d1539cd4a823e84ba5c732f22819 columnFamilyName family1 2023-03-22 15:54:17,538 INFO [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] regionserver.HStore(310): Store=4cb1d1539cd4a823e84ba5c732f22819/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:17,539 INFO [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:17,541 DEBUG [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819/family2 2023-03-22 15:54:17,541 DEBUG [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819/family2 2023-03-22 15:54:17,542 INFO [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4cb1d1539cd4a823e84ba5c732f22819 columnFamilyName family2 2023-03-22 15:54:17,543 INFO [StoreOpener-4cb1d1539cd4a823e84ba5c732f22819-1] regionserver.HStore(310): Store=4cb1d1539cd4a823e84ba5c732f22819/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:17,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:17,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:17,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:54:17,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:54:17,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:17,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4cb1d1539cd4a823e84ba5c732f22819; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70023457, jitterRate=0.04343082010746002}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:54:17,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4cb1d1539cd4a823e84ba5c732f22819: 2023-03-22 15:54:17,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819., pid=31, masterSystemTime=1679500457489 2023-03-22 15:54:17,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:17,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:17,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:17,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9183debe8c11b7195a71e3968c944c76, NAME => 'testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76.', STARTKEY => '2', ENDKEY => ''} 2023-03-22 15:54:17,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:17,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:17,557 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=4cb1d1539cd4a823e84ba5c732f22819, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:17,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:17,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:17,557 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679500457557"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500457557"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500457557"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500457557"}]},"ts":"1679500457557"} 2023-03-22 15:54:17,560 INFO [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:17,564 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-03-22 15:54:17,564 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure 4cb1d1539cd4a823e84ba5c732f22819, server=jenkins-hbase4.apache.org,38637,1679500416185 in 224 msec 2023-03-22 15:54:17,565 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4cb1d1539cd4a823e84ba5c732f22819, ASSIGN in 395 msec 2023-03-22 15:54:17,570 DEBUG [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1 2023-03-22 15:54:17,570 DEBUG [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1 2023-03-22 15:54:17,571 INFO [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9183debe8c11b7195a71e3968c944c76 columnFamilyName family1 2023-03-22 15:54:17,574 INFO [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] regionserver.HStore(310): Store=9183debe8c11b7195a71e3968c944c76/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:17,574 INFO [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:17,581 DEBUG [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family2 2023-03-22 15:54:17,583 DEBUG [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family2 2023-03-22 15:54:17,583 INFO [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9183debe8c11b7195a71e3968c944c76 columnFamilyName family2 2023-03-22 15:54:17,585 INFO [StoreOpener-9183debe8c11b7195a71e3968c944c76-1] regionserver.HStore(310): Store=9183debe8c11b7195a71e3968c944c76/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:17,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:17,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:17,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:54:17,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9183debe8c11b7195a71e3968c944c76 2023-03-22 15:54:17,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:17,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9183debe8c11b7195a71e3968c944c76; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74088948, jitterRate=0.10401135683059692}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:54:17,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9183debe8c11b7195a71e3968c944c76: 2023-03-22 15:54:17,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76., pid=32, masterSystemTime=1679500457489 2023-03-22 15:54:17,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:17,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:17,606 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=9183debe8c11b7195a71e3968c944c76, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:17,607 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679500457606"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500457606"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500457606"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500457606"}]},"ts":"1679500457606"} 2023-03-22 15:54:17,610 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-03-22 15:54:17,610 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure 9183debe8c11b7195a71e3968c944c76, server=jenkins-hbase4.apache.org,38637,1679500416185 in 271 msec 2023-03-22 15:54:17,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-03-22 15:54:17,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=9183debe8c11b7195a71e3968c944c76, ASSIGN in 441 msec 2023-03-22 15:54:17,613 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:54:17,613 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500457613"}]},"ts":"1679500457613"} 2023-03-22 15:54:17,615 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-22 15:54:17,618 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:54:17,622 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.3390 sec 2023-03-22 15:54:18,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-22 15:54:18,393 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-22 15:54:18,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:18,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:18,416 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-22 15:54:18,416 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:18,416 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-22 15:54:18,416 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-22 15:54:18,417 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:18,417 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:18,418 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:18,418 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:18,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:18,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:18,419 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:18,419 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:18,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:18,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:18,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:18,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:18,420 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:18,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-22 15:54:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-22 15:54:18,424 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:18,424 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. started... 2023-03-22 15:54:18,425 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c20178396b93c0ac2f980d2114beb404: 2023-03-22 15:54:18,425 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:54:18,425 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:18,425 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:18,425 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. started... 2023-03-22 15:54:18,425 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. started... 2023-03-22 15:54:18,425 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:18,425 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:18,426 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4cb1d1539cd4a823e84ba5c732f22819: 2023-03-22 15:54:18,426 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:54:18,426 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9183debe8c11b7195a71e3968c944c76 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-22 15:54:18,432 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:18,432 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:54:18,859 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322d2e700b2f27248fbb4a686d309f65904 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230322d2e700b2f27248fbb4a686d309f65904 2023-03-22 15:54:19,266 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/.tmp/family1/e5b028c2def845daa0004a4c929f63ca 2023-03-22 15:54:19,272 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/.tmp/family1/e5b028c2def845daa0004a4c929f63ca as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/e5b028c2def845daa0004a4c929f63ca 2023-03-22 15:54:19,279 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/e5b028c2def845daa0004a4c929f63ca, entries=1, sequenceid=5, filesize=4.9 K 2023-03-22 15:54:19,280 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 9183debe8c11b7195a71e3968c944c76 in 854ms, sequenceid=5, compaction requested=false 2023-03-22 15:54:19,280 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-22 15:54:19,281 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9183debe8c11b7195a71e3968c944c76: 2023-03-22 15:54:19,281 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:19,281 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:54:19,281 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:54:19,281 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,281 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-22 15:54:19,281 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-22 15:54:19,284 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,284 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:19,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:19,284 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,284 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:19,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:19,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:19,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:19,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-22 15:54:19,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23f0a06e[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:19,286 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-22 15:54:19,286 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,289 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,289 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-22 15:54:19,289 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-22 15:54:19,289 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:19,289 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,289 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:19,291 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,291 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-22 15:54:19,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:19,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:19,291 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:19,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-22 15:54:19,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:19,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:19,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:19,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:19,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-22 15:54:19,295 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-22 15:54:19,295 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:19,295 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:19,295 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,295 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:19,297 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,297 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,297 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:19,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:19,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:19,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:19,297 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:19,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:19,299 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:19,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:19,311 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:19,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:19,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:19,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-22 15:54:19,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:19,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-22 15:54:19,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:19,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:19,312 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:19,312 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:19,412 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:19,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:19,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:19,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:19,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,426 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-22 15:54:19,426 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:19,427 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-22 15:54:19,427 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-22 15:54:19,427 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,427 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,429 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:19,429 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:19,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:19,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,430 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,430 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:19,432 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:19,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,440 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,440 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-22 15:54:19,441 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,441 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-22 15:54:19,441 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:19,441 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-22 15:54:19,444 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:19,444 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. started... 2023-03-22 15:54:19,444 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c20178396b93c0ac2f980d2114beb404: 2023-03-22 15:54:19,444 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:54:19,445 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:19,445 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:54:19,445 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:19,445 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. started... 2023-03-22 15:54:19,445 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. started... 2023-03-22 15:54:19,445 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:19,446 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 4cb1d1539cd4a823e84ba5c732f22819: 2023-03-22 15:54:19,446 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:54:19,446 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9183debe8c11b7195a71e3968c944c76 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-22 15:54:19,446 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:54:19,446 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:54:19,480 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322dbc4d2fb79aa4755b851d43c58325a83 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230322dbc4d2fb79aa4755b851d43c58325a83 2023-03-22 15:54:19,522 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/.tmp/family1/620e7c1bccca443a8a9f3a529a9c222d 2023-03-22 15:54:19,530 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/.tmp/family1/620e7c1bccca443a8a9f3a529a9c222d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/620e7c1bccca443a8a9f3a529a9c222d 2023-03-22 15:54:19,539 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/620e7c1bccca443a8a9f3a529a9c222d, entries=2, sequenceid=9, filesize=5.0 K 2023-03-22 15:54:19,540 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 9183debe8c11b7195a71e3968c944c76 in 94ms, sequenceid=9, compaction requested=false 2023-03-22 15:54:19,541 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9183debe8c11b7195a71e3968c944c76: 2023-03-22 15:54:19,541 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:19,541 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:54:19,541 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:54:19,541 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,541 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-22 15:54:19,541 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-22 15:54:19,543 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,543 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:19,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:19,543 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,543 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:19,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:19,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:19,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:19,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-22 15:54:19,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@774d6b0b[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:19,547 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-22 15:54:19,547 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,549 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,549 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:19,549 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,549 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-22 15:54:19,550 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-22 15:54:19,550 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:19,552 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-22 15:54:19,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,552 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:19,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:19,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:19,552 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-22 15:54:19,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:19,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:19,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:19,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:19,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-22 15:54:19,555 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-22 15:54:19,555 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:19,555 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:19,555 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,555 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:19,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:19,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:19,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:19,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,558 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:19,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:19,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:19,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,559 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:19,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:19,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,561 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,562 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:19,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-22 15:54:19,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:19,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:19,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:19,562 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:19,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-22 15:54:19,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:19,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:19,662 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:19,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:19,672 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-22 15:54:19,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-22 15:54:19,675 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-22 15:54:19,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:19,676 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501059676 2023-03-22 15:54:19,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:54:19,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-22 15:54:19,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-22 15:54:19,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x2cc37329 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:19,682 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4574eee7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:19,683 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230322 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-22 15:54:20,534 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303223fcfa48af5ac4520be1926421af5da03 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303223fcfa48af5ac4520be1926421af5da03 2023-03-22 15:54:20,535 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-22 15:54:20,553 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:54:20,554 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42218, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:54:20,567 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:54:20,570 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60064, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:54:20,585 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230322/family1/0d4f48c9f84b4921b7e4b7c4cc28d4c2 first=Optional[k0] last=Optional[k1] 2023-03-22 15:54:20,585 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230322/family1/0d4f48c9f84b4921b7e4b7c4cc28d4c2}] 2023-03-22 15:54:20,591 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230322/family1/0d4f48c9f84b4921b7e4b7c4cc28d4c2 for inclusion in 9183debe8c11b7195a71e3968c944c76/family1 2023-03-22 15:54:20,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-22 15:54:20,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-22 15:54:20,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 9183debe8c11b7195a71e3968c944c76: 2023-03-22 15:54:20,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230322/family1/0d4f48c9f84b4921b7e4b7c4cc28d4c2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testScannerOnBulkLoadRefHFiles__rmf5s0m5i3klj6qbpp25uq0pj3fgphj7peov6dciigooqui5qotvga7ncnqhi95p/family1/0d4f48c9f84b4921b7e4b7c4cc28d4c2 2023-03-22 15:54:20,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testScannerOnBulkLoadRefHFiles__rmf5s0m5i3klj6qbpp25uq0pj3fgphj7peov6dciigooqui5qotvga7ncnqhi95p/family1/0d4f48c9f84b4921b7e4b7c4cc28d4c2 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/0c490515c1cc4ccd826a824b1419ec47_SeqId_12_ 2023-03-22 15:54:20,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testScannerOnBulkLoadRefHFiles__rmf5s0m5i3klj6qbpp25uq0pj3fgphj7peov6dciigooqui5qotvga7ncnqhi95p/family1/0d4f48c9f84b4921b7e4b7c4cc28d4c2 into 9183debe8c11b7195a71e3968c944c76/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/0c490515c1cc4ccd826a824b1419ec47_SeqId_12_ - updating store file list. 2023-03-22 15:54:20,604 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/0c490515c1cc4ccd826a824b1419ec47_SeqId_12_ into 9183debe8c11b7195a71e3968c944c76/family1 2023-03-22 15:54:20,604 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testScannerOnBulkLoadRefHFiles__rmf5s0m5i3klj6qbpp25uq0pj3fgphj7peov6dciigooqui5qotvga7ncnqhi95p/family1/0d4f48c9f84b4921b7e4b7c4cc28d4c2 into 9183debe8c11b7195a71e3968c944c76/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/0c490515c1cc4ccd826a824b1419ec47_SeqId_12_) 2023-03-22 15:54:20,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testScannerOnBulkLoadRefHFiles__rmf5s0m5i3klj6qbpp25uq0pj3fgphj7peov6dciigooqui5qotvga7ncnqhi95p/family1/0d4f48c9f84b4921b7e4b7c4cc28d4c2 2023-03-22 15:54:20,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-22 15:54:20,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-22 15:54:20,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): 9183debe8c11b7195a71e3968c944c76/family1 is initiating minor compaction (all files) 2023-03-22 15:54:20,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@746a8aa8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:20,606 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9183debe8c11b7195a71e3968c944c76/family1 in testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:54:20,606 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/e5b028c2def845daa0004a4c929f63ca, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/620e7c1bccca443a8a9f3a529a9c222d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/0c490515c1cc4ccd826a824b1419ec47_SeqId_12_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/.tmp, totalSize=15.1 K 2023-03-22 15:54:20,607 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting e5b028c2def845daa0004a4c929f63ca, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500458393 2023-03-22 15:54:20,608 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 620e7c1bccca443a8a9f3a529a9c222d, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500458393 2023-03-22 15:54:20,610 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 0c490515c1cc4ccd826a824b1419ec47_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500458393 2023-03-22 15:54:20,610 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-22 15:54:20,612 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:20,615 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230322d2e700b2f27248fbb4a686d309f65904 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230322d2e700b2f27248fbb4a686d309f65904 2023-03-22 15:54:20,616 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230322dbc4d2fb79aa4755b851d43c58325a83 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230322dbc4d2fb79aa4755b851d43c58325a83 2023-03-22 15:54:20,616 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-22 15:54:20,616 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230322' 2023-03-22 15:54:20,617 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230322 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-22 15:54:20,617 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-22 15:54:20,617 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:20,618 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-22 15:54:20,623 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 945 msec 2023-03-22 15:54:20,639 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9183debe8c11b7195a71e3968c944c76#family1#compaction#37 average throughput is 0.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:54:20,657 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=253 (was 257), OpenFileDescriptor=614 (was 616), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=446 (was 433) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6663 (was 6708) 2023-03-22 15:54:20,671 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=252, OpenFileDescriptor=614, MaxFileDescriptor=60000, SystemLoadAverage=446, ProcessCount=170, AvailableMemoryMB=6663 2023-03-22 15:54:20,673 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-22 15:54:20,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:20,676 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:54:20,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-22 15:54:20,677 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:54:20,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-22 15:54:20,679 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616 2023-03-22 15:54:20,680 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616 empty. 2023-03-22 15:54:20,680 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616 2023-03-22 15:54:20,680 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-22 15:54:20,694 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-22 15:54:20,695 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 655cd9582db7f9e343db22d211baf616, NAME => 'testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:20,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-22 15:54:20,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-22 15:54:21,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:21,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 655cd9582db7f9e343db22d211baf616, disabling compactions & flushes 2023-03-22 15:54:21,103 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:21,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:21,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. after waiting 0 ms 2023-03-22 15:54:21,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:21,103 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:21,103 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:21,106 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:54:21,107 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679500461107"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500461107"}]},"ts":"1679500461107"} 2023-03-22 15:54:21,109 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-22 15:54:21,110 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:54:21,110 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500461110"}]},"ts":"1679500461110"} 2023-03-22 15:54:21,111 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-22 15:54:21,116 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=655cd9582db7f9e343db22d211baf616, ASSIGN}] 2023-03-22 15:54:21,117 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=655cd9582db7f9e343db22d211baf616, ASSIGN 2023-03-22 15:54:21,118 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=655cd9582db7f9e343db22d211baf616, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:21,269 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=655cd9582db7f9e343db22d211baf616, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:21,269 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679500461269"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500461269"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500461269"}]},"ts":"1679500461269"} 2023-03-22 15:54:21,271 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 655cd9582db7f9e343db22d211baf616, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:21,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-22 15:54:21,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:21,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 655cd9582db7f9e343db22d211baf616, NAME => 'testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:54:21,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 655cd9582db7f9e343db22d211baf616 2023-03-22 15:54:21,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:21,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 655cd9582db7f9e343db22d211baf616 2023-03-22 15:54:21,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 655cd9582db7f9e343db22d211baf616 2023-03-22 15:54:21,429 INFO [StoreOpener-655cd9582db7f9e343db22d211baf616-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 655cd9582db7f9e343db22d211baf616 2023-03-22 15:54:21,430 DEBUG [StoreOpener-655cd9582db7f9e343db22d211baf616-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1 2023-03-22 15:54:21,430 DEBUG [StoreOpener-655cd9582db7f9e343db22d211baf616-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1 2023-03-22 15:54:21,430 INFO [StoreOpener-655cd9582db7f9e343db22d211baf616-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 655cd9582db7f9e343db22d211baf616 columnFamilyName family1 2023-03-22 15:54:21,432 INFO [StoreOpener-655cd9582db7f9e343db22d211baf616-1] regionserver.HStore(310): Store=655cd9582db7f9e343db22d211baf616/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:21,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616 2023-03-22 15:54:21,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616 2023-03-22 15:54:21,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 655cd9582db7f9e343db22d211baf616 2023-03-22 15:54:21,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:21,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 655cd9582db7f9e343db22d211baf616; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63133348, jitterRate=-0.05923980474472046}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:54:21,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:21,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616., pid=37, masterSystemTime=1679500461423 2023-03-22 15:54:21,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:21,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:21,444 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=655cd9582db7f9e343db22d211baf616, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:21,445 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679500461444"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500461444"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500461444"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500461444"}]},"ts":"1679500461444"} 2023-03-22 15:54:21,448 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-22 15:54:21,448 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 655cd9582db7f9e343db22d211baf616, server=jenkins-hbase4.apache.org,38637,1679500416185 in 175 msec 2023-03-22 15:54:21,451 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-22 15:54:21,451 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=655cd9582db7f9e343db22d211baf616, ASSIGN in 333 msec 2023-03-22 15:54:21,451 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:54:21,452 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500461451"}]},"ts":"1679500461451"} 2023-03-22 15:54:21,453 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-22 15:54:21,455 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:54:21,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 782 msec 2023-03-22 15:54:21,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-22 15:54:21,782 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-22 15:54:21,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:21,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:21,816 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:21,816 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:21,816 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:21,816 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:21,817 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:21,817 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:21,819 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:21,819 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:21,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:21,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:21,819 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:21,819 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:21,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:21,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:21,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:21,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:21,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:21,821 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:21,821 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:21,821 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:21,821 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:21,821 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:21,821 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:21,821 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:21,821 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:21,822 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-22 15:54:21,874 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/.tmp/family1/aab5e11d5cf247c7aa4455b1f44e2605 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/aab5e11d5cf247c7aa4455b1f44e2605 2023-03-22 15:54:21,880 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 9183debe8c11b7195a71e3968c944c76/family1 of 9183debe8c11b7195a71e3968c944c76 into aab5e11d5cf247c7aa4455b1f44e2605(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:54:21,880 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9183debe8c11b7195a71e3968c944c76: 2023-03-22 15:54:21,880 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1679500460606; duration=1sec 2023-03-22 15:54:21,880 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:21,899 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/e5b028c2def845daa0004a4c929f63ca, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/620e7c1bccca443a8a9f3a529a9c222d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/0c490515c1cc4ccd826a824b1419ec47_SeqId_12_] to archive 2023-03-22 15:54:21,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:21,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/e5b028c2def845daa0004a4c929f63ca to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/e5b028c2def845daa0004a4c929f63ca 2023-03-22 15:54:21,903 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/620e7c1bccca443a8a9f3a529a9c222d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/620e7c1bccca443a8a9f3a529a9c222d 2023-03-22 15:54:21,904 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/0c490515c1cc4ccd826a824b1419ec47_SeqId_12_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/family1/0c490515c1cc4ccd826a824b1419ec47_SeqId_12_ 2023-03-22 15:54:22,263 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130edead54438364e62960064c9fcd144a0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130edead54438364e62960064c9fcd144a0 2023-03-22 15:54:22,267 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/f1f034c4f0304175994f783c1e756af7 2023-03-22 15:54:22,273 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/f1f034c4f0304175994f783c1e756af7 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/f1f034c4f0304175994f783c1e756af7 2023-03-22 15:54:22,277 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/f1f034c4f0304175994f783c1e756af7, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-22 15:54:22,278 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 456ms, sequenceid=5, compaction requested=false 2023-03-22 15:54:22,278 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:22,279 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:22,279 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:22,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:22,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:22,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:22,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:22,282 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,282 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:22,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:22,282 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,282 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:22,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:22,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:22,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:22,284 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:22,284 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@629abd7f[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:22,284 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:22,284 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,285 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:22,286 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,286 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:22,286 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:22,287 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:22,287 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,287 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:22,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:22,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:22,288 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:22,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:22,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:22,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:22,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:22,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:22,290 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:22,290 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:22,290 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:22,290 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,290 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:22,292 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,292 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:22,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:22,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,292 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:22,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:22,292 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:22,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:22,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:22,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,294 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,297 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,297 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:22,297 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:22,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:22,297 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:22,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:22,297 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:22,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:22,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:22,297 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:22,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:22,298 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,298 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,298 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,298 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,298 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:22,298 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:22,398 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:22,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:22,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:22,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:22,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,406 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:22,406 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:22,406 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:22,406 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:22,407 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,407 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,409 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,409 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:22,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:22,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:22,409 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:22,409 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:22,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:22,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,410 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,410 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:22,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:22,411 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:22,411 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:22,411 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:22,411 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:22,411 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:22,411 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:22,411 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:22,411 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-22 15:54:22,829 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128260f3702ed474c2ba7e42a5d740f6b36 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128260f3702ed474c2ba7e42a5d740f6b36 2023-03-22 15:54:23,042 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:54:23,238 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/8210ba06347842d7bc53a80c24d3ea9f 2023-03-22 15:54:23,245 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/8210ba06347842d7bc53a80c24d3ea9f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/8210ba06347842d7bc53a80c24d3ea9f 2023-03-22 15:54:23,251 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/8210ba06347842d7bc53a80c24d3ea9f, entries=1, sequenceid=9, filesize=5.0 K 2023-03-22 15:54:23,252 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 841ms, sequenceid=9, compaction requested=false 2023-03-22 15:54:23,252 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:23,252 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:23,252 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:23,252 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:23,252 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,253 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:23,253 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:23,255 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,255 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:23,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:23,255 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,256 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:23,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:23,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:23,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:23,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:23,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b14c37e[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:23,258 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:23,258 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,263 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,263 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:23,263 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:23,263 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:23,265 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:23,265 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:23,265 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:23,269 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,269 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:23,269 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:23,269 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:23,269 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:23,269 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:23,269 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,269 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:23,271 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:23,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:23,271 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,271 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:23,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:23,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:23,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:23,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,272 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:23,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,274 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:23,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,277 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,277 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,277 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,277 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,277 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:23,277 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:23,277 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,277 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:23,277 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:23,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:23,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:23,279 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:23,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:23,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:23,280 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:23,280 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:23,381 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:23,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:23,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:23,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:23,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,391 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:23,391 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:23,391 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:23,391 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:23,391 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,391 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,394 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:23,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:23,394 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:23,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,395 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,395 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:23,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:23,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:23,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,404 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:23,404 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:23,404 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:23,404 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:23,404 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:23,404 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:23,404 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:23,404 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:23,832 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512014ade625ec8834ba686b91c2b17bf13b6 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512014ade625ec8834ba686b91c2b17bf13b6 2023-03-22 15:54:23,857 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/4eba7cd0c81f472f8edf3f4f491e1836 2023-03-22 15:54:23,865 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/4eba7cd0c81f472f8edf3f4f491e1836 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4eba7cd0c81f472f8edf3f4f491e1836 2023-03-22 15:54:23,871 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4eba7cd0c81f472f8edf3f4f491e1836, entries=1, sequenceid=13, filesize=4.9 K 2023-03-22 15:54:23,873 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 469ms, sequenceid=13, compaction requested=true 2023-03-22 15:54:23,873 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:23,873 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:23,873 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:23,873 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:23,873 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,873 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:23,873 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:23,875 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,875 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:23,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:23,875 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,875 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:23,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:23,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:23,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:23,879 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:23,879 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dc1e3a2[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:23,879 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:23,879 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,881 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,881 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:23,881 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:23,881 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,881 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:23,881 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:23,885 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:23,885 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:23,885 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:23,885 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:23,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:23,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:23,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:23,891 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:23,891 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:23,891 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:23,891 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,891 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:23,893 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:23,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:23,893 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,893 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:23,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:23,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:23,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:23,894 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:23,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,896 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:23,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,898 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,898 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,898 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,898 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:23,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:23,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:23,898 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:23,898 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,898 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:23,898 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:23,898 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,898 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:23,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:23,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:23,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:23,899 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:23,899 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:23,999 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:24,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:24,004 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:24,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:24,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,006 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:24,006 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:24,006 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:24,006 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:24,007 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,007 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,008 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:24,008 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:24,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:24,009 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,009 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:24,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,010 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:24,010 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,010 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:24,010 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:24,010 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:24,010 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:24,010 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:24,010 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:24,010 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:24,010 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:24,435 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205cd439f363d4940a78da96f6fcc02f817 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205cd439f363d4940a78da96f6fcc02f817 2023-03-22 15:54:24,842 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/e8635f4ec68a4be98c16ce7b97ca2cfb 2023-03-22 15:54:24,848 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/e8635f4ec68a4be98c16ce7b97ca2cfb as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/e8635f4ec68a4be98c16ce7b97ca2cfb 2023-03-22 15:54:24,856 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/e8635f4ec68a4be98c16ce7b97ca2cfb, entries=1, sequenceid=17, filesize=4.9 K 2023-03-22 15:54:24,857 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 847ms, sequenceid=17, compaction requested=true 2023-03-22 15:54:24,857 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:24,857 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:24,857 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:24,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:24,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:24,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:24,860 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,860 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:24,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:24,861 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,861 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:24,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:24,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:24,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:24,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2148c222[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:24,863 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:24,863 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,864 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,864 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,864 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:24,865 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:24,865 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:24,865 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:24,866 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:24,866 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:24,866 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,866 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:24,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:24,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:24,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:24,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:24,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:24,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:24,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:24,870 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:24,870 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:24,870 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:24,870 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,870 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:24,871 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:24,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:24,872 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,872 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:24,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:24,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:24,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:24,872 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:24,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,874 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:24,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,878 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,878 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,878 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,878 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,878 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,879 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:24,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:24,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:24,878 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:24,879 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:24,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:24,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:24,879 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:24,879 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:24,879 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,879 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:24,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:24,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:24,979 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:24,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:24,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:24,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:24,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,992 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:24,992 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:24,992 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:24,992 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:24,992 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,993 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,994 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:24,994 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:24,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:24,995 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:24,995 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:24,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:24,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:24,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:24,996 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:24,996 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:24,996 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:24,996 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:24,996 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:24,996 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:24,996 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:24,996 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:25,416 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512289853d6f55185451480eb5cefea083181 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512289853d6f55185451480eb5cefea083181 2023-03-22 15:54:25,436 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/4b6bfc1c9e8440458b0eff98bf49ffc8 2023-03-22 15:54:25,443 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/4b6bfc1c9e8440458b0eff98bf49ffc8 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4b6bfc1c9e8440458b0eff98bf49ffc8 2023-03-22 15:54:25,447 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4b6bfc1c9e8440458b0eff98bf49ffc8, entries=1, sequenceid=21, filesize=4.9 K 2023-03-22 15:54:25,448 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 452ms, sequenceid=21, compaction requested=true 2023-03-22 15:54:25,448 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:25,448 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:25,448 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:25,448 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:25,448 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,448 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:25,448 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:25,452 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,452 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,453 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,453 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:25,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:25,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@160ee231[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:25,458 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:25,458 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,459 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,459 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:25,460 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:25,460 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,460 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:25,461 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,461 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:25,461 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:25,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,461 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:25,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:25,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:25,464 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:25,464 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:25,464 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:25,464 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,464 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:25,468 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,468 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,468 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:25,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:25,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:25,468 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,470 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,472 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:25,472 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:25,472 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:25,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:25,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:25,472 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:25,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:25,472 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,472 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:25,473 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:25,473 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:25,473 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,473 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,473 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,473 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,473 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:25,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:25,573 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:25,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:25,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:25,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:25,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,580 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:25,580 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:25,580 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:25,580 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:25,580 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,580 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,582 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:25,582 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:25,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:25,582 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,582 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:25,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:25,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,583 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,583 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:25,583 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:25,584 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:25,584 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:25,584 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:25,598 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123157916bef2e3f4c16bc0153b1f0f76052 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123157916bef2e3f4c16bc0153b1f0f76052 2023-03-22 15:54:25,603 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/18eced02462b40c8a5f19abc4dfdfdae 2023-03-22 15:54:25,608 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/18eced02462b40c8a5f19abc4dfdfdae as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/18eced02462b40c8a5f19abc4dfdfdae 2023-03-22 15:54:25,613 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/18eced02462b40c8a5f19abc4dfdfdae, entries=1, sequenceid=25, filesize=4.9 K 2023-03-22 15:54:25,614 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 30ms, sequenceid=25, compaction requested=true 2023-03-22 15:54:25,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:25,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:25,616 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,616 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:25,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bbcccbd[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:25,618 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:25,618 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,620 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,620 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:25,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:25,620 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:25,620 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:25,621 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,621 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,621 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:25,622 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:25,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:25,624 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:25,624 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:25,624 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:25,624 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,624 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:25,625 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,625 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:25,626 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:25,626 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,627 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,629 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:25,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:25,630 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:25,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,630 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:25,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:25,730 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:25,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:25,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:25,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:25,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,739 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:25,739 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:25,739 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:25,739 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:25,740 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,740 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,742 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:25,742 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:25,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:25,742 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,742 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:25,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:25,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,743 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,743 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:25,743 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,743 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:25,744 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:25,744 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:25,744 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:25,744 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:25,744 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:25,744 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:25,744 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:25,769 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101f3656f5e1fdf489b9befb6c6a07eb4e5 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101f3656f5e1fdf489b9befb6c6a07eb4e5 2023-03-22 15:54:25,787 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/80a65274a40945ca8db2ce1830de8538 2023-03-22 15:54:25,794 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/80a65274a40945ca8db2ce1830de8538 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/80a65274a40945ca8db2ce1830de8538 2023-03-22 15:54:25,800 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/80a65274a40945ca8db2ce1830de8538, entries=1, sequenceid=29, filesize=4.9 K 2023-03-22 15:54:25,801 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 57ms, sequenceid=29, compaction requested=true 2023-03-22 15:54:25,801 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:25,801 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:25,801 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:25,801 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:25,801 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,801 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:25,801 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:25,803 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,803 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,804 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,804 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:25,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:25,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1952e69b[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:25,805 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:25,805 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,807 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,807 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:25,807 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:25,807 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:25,807 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,807 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:25,809 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:25,809 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,809 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:25,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,809 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:25,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:25,812 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:25,812 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:25,812 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:25,812 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,812 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:25,817 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,817 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,817 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,818 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,820 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,823 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:25,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:25,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:25,823 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:25,824 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:25,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:25,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:25,924 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:25,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:25,930 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:25,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:25,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,932 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:25,932 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:25,932 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:25,932 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:25,932 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,932 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,936 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,936 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:25,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:25,937 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,937 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:25,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:25,939 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:25,939 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:25,939 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:25,939 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:25,939 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:25,939 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:25,966 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103731541b9900847069fb5bc4747bc49fb to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103731541b9900847069fb5bc4747bc49fb 2023-03-22 15:54:25,978 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/d6c1f4643ae3412384126be2d915ce9f 2023-03-22 15:54:25,984 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/d6c1f4643ae3412384126be2d915ce9f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6c1f4643ae3412384126be2d915ce9f 2023-03-22 15:54:25,988 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6c1f4643ae3412384126be2d915ce9f, entries=1, sequenceid=33, filesize=4.9 K 2023-03-22 15:54:25,989 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 50ms, sequenceid=33, compaction requested=true 2023-03-22 15:54:25,989 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:25,989 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:25,991 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,991 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,991 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,991 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:25,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:25,993 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:25,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7eec6e41[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:25,993 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,995 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,995 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:25,995 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:25,995 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:25,995 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,995 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:25,998 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:25,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:25,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:25,998 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:25,998 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:25,998 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:26,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:26,002 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:26,002 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:26,002 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:26,002 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:26,002 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,002 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:26,003 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,003 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:26,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:26,003 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:26,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:26,004 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:26,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:26,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:26,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,005 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:26,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,008 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:26,008 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,008 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:26,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:26,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:26,008 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,009 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,009 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,009 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:26,009 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,009 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:26,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:26,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:26,009 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,009 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:26,010 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:26,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:26,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:26,110 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:26,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:26,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:26,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:26,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,117 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:26,117 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:26,117 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:26,117 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:26,118 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,118 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,119 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:26,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:26,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:26,119 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,119 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:26,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:26,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,121 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:26,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,121 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:26,121 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:26,121 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:26,126 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:26,126 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:26,127 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:26,126 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:26,127 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:26,145 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509071f8db0438fef4778ab65e5ee6e9dd161 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201509071f8db0438fef4778ab65e5ee6e9dd161 2023-03-22 15:54:26,167 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/2b755c8025554adca6200936f695c370 2023-03-22 15:54:26,173 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/2b755c8025554adca6200936f695c370 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/2b755c8025554adca6200936f695c370 2023-03-22 15:54:26,177 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/2b755c8025554adca6200936f695c370, entries=1, sequenceid=37, filesize=4.9 K 2023-03-22 15:54:26,178 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 51ms, sequenceid=37, compaction requested=true 2023-03-22 15:54:26,178 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:26,178 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:26,178 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:26,178 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:26,178 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,179 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:26,179 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:26,182 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,182 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:26,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:26,182 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,182 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:26,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:26,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:26,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:26,184 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:26,184 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:26,184 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f1204c7[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:26,184 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,185 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,185 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:26,185 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,185 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:26,185 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:26,185 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:26,187 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:26,187 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,187 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:26,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:26,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:26,187 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:26,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:26,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:26,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:26,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:26,189 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:26,189 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:26,189 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:26,189 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:26,189 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,189 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:26,191 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,191 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:26,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,191 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:26,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:26,192 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:26,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,193 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:26,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,197 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,197 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:26,197 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:26,197 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:26,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:26,197 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,197 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,197 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:26,197 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:26,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:26,197 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,197 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,198 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:26,198 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:26,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:26,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:26,298 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:26,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:26,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:26,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:26,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,305 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:26,305 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:26,305 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-22 15:54:26,305 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-22 15:54:26,305 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,305 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,307 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,307 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:26,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:26,307 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,307 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:26,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:26,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-22 15:54:26,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,309 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-22 15:54:26,309 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:26,309 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-22 15:54:26,309 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:26,309 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:26,309 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:26,309 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. started... 2023-03-22 15:54:26,309 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 655cd9582db7f9e343db22d211baf616 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:26,725 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112097bd75d07d88446ebcd06bd1650ad49d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112097bd75d07d88446ebcd06bd1650ad49d 2023-03-22 15:54:26,732 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/d6d8d30882bb4e03accdc0ae94144f32 2023-03-22 15:54:26,743 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/d6d8d30882bb4e03accdc0ae94144f32 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6d8d30882bb4e03accdc0ae94144f32 2023-03-22 15:54:26,749 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6d8d30882bb4e03accdc0ae94144f32, entries=1, sequenceid=41, filesize=4.9 K 2023-03-22 15:54:26,750 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 655cd9582db7f9e343db22d211baf616 in 441ms, sequenceid=41, compaction requested=true 2023-03-22 15:54:26,750 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:26,750 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:26,750 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:26,750 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:26,750 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,750 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-22 15:54:26,750 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-22 15:54:26,752 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,752 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:26,753 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,753 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:26,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:26,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-22 15:54:26,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5206fabc[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:26,754 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-22 15:54:26,754 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,757 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-22 15:54:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-22 15:54:26,757 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:26,758 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,758 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:26,759 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-22 15:54:26,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:26,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-22 15:54:26,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:26,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:26,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:26,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:26,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:54:26,761 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-22 15:54:26,761 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:26,761 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:26,761 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,761 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:26,763 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,763 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:26,763 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:26,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:26,764 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,765 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,771 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,771 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:26,771 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:26,771 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,772 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:26,771 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:26,772 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-22 15:54:26,772 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,772 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:26,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:26,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-22 15:54:26,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:26,772 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,772 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,772 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:26,772 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:26,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:26,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:26,873 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:26,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:26,875 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-22 15:54:26,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:54:26,877 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:54:26,878 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:26,878 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501066878 2023-03-22 15:54:26,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:54:26,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-22 15:54:26,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:26,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x6a3434be to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:26,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@8d5ecc9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:26,886 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,337 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c11e1455b37d4dc6b3fb964d12d8e89c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103c11e1455b37d4dc6b3fb964d12d8e89c 2023-03-22 15:54:27,337 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:27,356 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:54:27,357 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34168, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:54:27,364 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:54:27,365 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60726, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:54:27,376 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7a30101522a44883bbde9d74a62e79db first=Optional[r3] last=Optional[r6] 2023-03-22 15:54:27,377 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7a30101522a44883bbde9d74a62e79db}] 2023-03-22 15:54:27,382 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7a30101522a44883bbde9d74a62e79db for inclusion in 655cd9582db7f9e343db22d211baf616/family1 2023-03-22 15:54:27,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-22 15:54:27,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:27,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:27,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/7a30101522a44883bbde9d74a62e79db to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__b6r5558e1uceu3sr3e2cg00spkh95n5cda1lcqs4celrnvrvg23ngs5jdmct6jhh/family1/7a30101522a44883bbde9d74a62e79db 2023-03-22 15:54:27,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__b6r5558e1uceu3sr3e2cg00spkh95n5cda1lcqs4celrnvrvg23ngs5jdmct6jhh/family1/7a30101522a44883bbde9d74a62e79db as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43a1201e2b79497197a7be92adb4f470_SeqId_44_ 2023-03-22 15:54:27,389 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__b6r5558e1uceu3sr3e2cg00spkh95n5cda1lcqs4celrnvrvg23ngs5jdmct6jhh/family1/7a30101522a44883bbde9d74a62e79db into 655cd9582db7f9e343db22d211baf616/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43a1201e2b79497197a7be92adb4f470_SeqId_44_ - updating store file list. 2023-03-22 15:54:27,394 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43a1201e2b79497197a7be92adb4f470_SeqId_44_ into 655cd9582db7f9e343db22d211baf616/family1 2023-03-22 15:54:27,394 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__b6r5558e1uceu3sr3e2cg00spkh95n5cda1lcqs4celrnvrvg23ngs5jdmct6jhh/family1/7a30101522a44883bbde9d74a62e79db into 655cd9582db7f9e343db22d211baf616/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43a1201e2b79497197a7be92adb4f470_SeqId_44_) 2023-03-22 15:54:27,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__b6r5558e1uceu3sr3e2cg00spkh95n5cda1lcqs4celrnvrvg23ngs5jdmct6jhh/family1/7a30101522a44883bbde9d74a62e79db 2023-03-22 15:54:27,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-22 15:54:27,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-22 15:54:27,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): 655cd9582db7f9e343db22d211baf616/family1 is initiating minor compaction 2023-03-22 15:54:27,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1a8c58e6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:27,398 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 655cd9582db7f9e343db22d211baf616/family1 in testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:27,398 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/8210ba06347842d7bc53a80c24d3ea9f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4eba7cd0c81f472f8edf3f4f491e1836, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/e8635f4ec68a4be98c16ce7b97ca2cfb, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4b6bfc1c9e8440458b0eff98bf49ffc8, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/18eced02462b40c8a5f19abc4dfdfdae, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/80a65274a40945ca8db2ce1830de8538, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6c1f4643ae3412384126be2d915ce9f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/2b755c8025554adca6200936f695c370, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6d8d30882bb4e03accdc0ae94144f32, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43a1201e2b79497197a7be92adb4f470_SeqId_44_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp, totalSize=50.0 K 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 8210ba06347842d7bc53a80c24d3ea9f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 4eba7cd0c81f472f8edf3f4f491e1836, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting e8635f4ec68a4be98c16ce7b97ca2cfb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 4b6bfc1c9e8440458b0eff98bf49ffc8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 18eced02462b40c8a5f19abc4dfdfdae, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 80a65274a40945ca8db2ce1830de8538, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting d6c1f4643ae3412384126be2d915ce9f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 2b755c8025554adca6200936f695c370, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting d6d8d30882bb4e03accdc0ae94144f32, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-22 15:54:27,399 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 43a1201e2b79497197a7be92adb4f470_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-22 15:54:27,401 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:27,403 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:27,405 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512289853d6f55185451480eb5cefea083181 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512289853d6f55185451480eb5cefea083181 2023-03-22 15:54:27,406 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123157916bef2e3f4c16bc0153b1f0f76052 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123157916bef2e3f4c16bc0153b1f0f76052 2023-03-22 15:54:27,408 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101f3656f5e1fdf489b9befb6c6a07eb4e5 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101f3656f5e1fdf489b9befb6c6a07eb4e5 2023-03-22 15:54:27,412 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103731541b9900847069fb5bc4747bc49fb to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103731541b9900847069fb5bc4747bc49fb 2023-03-22 15:54:27,412 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-22 15:54:27,412 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-22 15:54:27,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,413 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,413 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,413 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,413 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,413 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,414 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,414 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,462 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): 655cd9582db7f9e343db22d211baf616#family1#compaction#48 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:54:27,880 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/43b4bdd19a8947e89a436c8d7e71450b as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43b4bdd19a8947e89a436c8d7e71450b 2023-03-22 15:54:27,886 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 655cd9582db7f9e343db22d211baf616/family1 of 655cd9582db7f9e343db22d211baf616 into 43b4bdd19a8947e89a436c8d7e71450b(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:27,886 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:27,886 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1679500467398; duration=0sec 2023-03-22 15:54:27,886 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:27,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/8210ba06347842d7bc53a80c24d3ea9f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4eba7cd0c81f472f8edf3f4f491e1836, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/e8635f4ec68a4be98c16ce7b97ca2cfb, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4b6bfc1c9e8440458b0eff98bf49ffc8, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/18eced02462b40c8a5f19abc4dfdfdae, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/80a65274a40945ca8db2ce1830de8538, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6c1f4643ae3412384126be2d915ce9f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/2b755c8025554adca6200936f695c370, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6d8d30882bb4e03accdc0ae94144f32, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43a1201e2b79497197a7be92adb4f470_SeqId_44_] to archive 2023-03-22 15:54:27,903 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:27,905 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/8210ba06347842d7bc53a80c24d3ea9f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/8210ba06347842d7bc53a80c24d3ea9f 2023-03-22 15:54:27,907 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4eba7cd0c81f472f8edf3f4f491e1836 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4eba7cd0c81f472f8edf3f4f491e1836 2023-03-22 15:54:27,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/e8635f4ec68a4be98c16ce7b97ca2cfb to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/e8635f4ec68a4be98c16ce7b97ca2cfb 2023-03-22 15:54:27,909 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205af6b3227d45a41418552d78f7222b05e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205af6b3227d45a41418552d78f7222b05e 2023-03-22 15:54:27,909 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4b6bfc1c9e8440458b0eff98bf49ffc8 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/4b6bfc1c9e8440458b0eff98bf49ffc8 2023-03-22 15:54:27,910 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:27,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/18eced02462b40c8a5f19abc4dfdfdae to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/18eced02462b40c8a5f19abc4dfdfdae 2023-03-22 15:54:27,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/80a65274a40945ca8db2ce1830de8538 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/80a65274a40945ca8db2ce1830de8538 2023-03-22 15:54:27,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6c1f4643ae3412384126be2d915ce9f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6c1f4643ae3412384126be2d915ce9f 2023-03-22 15:54:27,916 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/2b755c8025554adca6200936f695c370 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/2b755c8025554adca6200936f695c370 2023-03-22 15:54:27,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6d8d30882bb4e03accdc0ae94144f32 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/d6d8d30882bb4e03accdc0ae94144f32 2023-03-22 15:54:27,919 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43a1201e2b79497197a7be92adb4f470_SeqId_44_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43a1201e2b79497197a7be92adb4f470_SeqId_44_ 2023-03-22 15:54:27,950 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7ca3f1327e4f47f9a8e857575f81d337 first=Optional[r00] last=Optional[r2] 2023-03-22 15:54:27,951 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7ca3f1327e4f47f9a8e857575f81d337}] 2023-03-22 15:54:27,954 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7ca3f1327e4f47f9a8e857575f81d337 for inclusion in 655cd9582db7f9e343db22d211baf616/family1 2023-03-22 15:54:27,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-22 15:54:27,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:27,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:27,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7ca3f1327e4f47f9a8e857575f81d337 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mogbndfg294stanhfmljtcrrd0900ch56og16ovtvbui0cke1v4n57q9ercvnhth/family1/7ca3f1327e4f47f9a8e857575f81d337 2023-03-22 15:54:27,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mogbndfg294stanhfmljtcrrd0900ch56og16ovtvbui0cke1v4n57q9ercvnhth/family1/7ca3f1327e4f47f9a8e857575f81d337 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/0aae96f8e4e24525a4a233c31e9a1136_SeqId_47_ 2023-03-22 15:54:27,963 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mogbndfg294stanhfmljtcrrd0900ch56og16ovtvbui0cke1v4n57q9ercvnhth/family1/7ca3f1327e4f47f9a8e857575f81d337 into 655cd9582db7f9e343db22d211baf616/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/0aae96f8e4e24525a4a233c31e9a1136_SeqId_47_ - updating store file list. 2023-03-22 15:54:27,967 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/0aae96f8e4e24525a4a233c31e9a1136_SeqId_47_ into 655cd9582db7f9e343db22d211baf616/family1 2023-03-22 15:54:27,967 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mogbndfg294stanhfmljtcrrd0900ch56og16ovtvbui0cke1v4n57q9ercvnhth/family1/7ca3f1327e4f47f9a8e857575f81d337 into 655cd9582db7f9e343db22d211baf616/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/0aae96f8e4e24525a4a233c31e9a1136_SeqId_47_) 2023-03-22 15:54:27,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mogbndfg294stanhfmljtcrrd0900ch56og16ovtvbui0cke1v4n57q9ercvnhth/family1/7ca3f1327e4f47f9a8e857575f81d337 2023-03-22 15:54:27,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-22 15:54:27,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-22 15:54:27,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): 655cd9582db7f9e343db22d211baf616/family1 is initiating minor compaction (all files) 2023-03-22 15:54:27,970 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 655cd9582db7f9e343db22d211baf616/family1 in testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:54:27,970 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/f1f034c4f0304175994f783c1e756af7, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43b4bdd19a8947e89a436c8d7e71450b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/0aae96f8e4e24525a4a233c31e9a1136_SeqId_47_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp, totalSize=337.0 K 2023-03-22 15:54:27,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@74f6a78f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:27,970 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting f1f034c4f0304175994f783c1e756af7, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840482 2023-03-22 15:54:27,971 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 43b4bdd19a8947e89a436c8d7e71450b, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240482 2023-03-22 15:54:27,971 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 0aae96f8e4e24525a4a233c31e9a1136_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840482 2023-03-22 15:54:27,973 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:27,974 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:27,977 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130edead54438364e62960064c9fcd144a0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130edead54438364e62960064c9fcd144a0 2023-03-22 15:54:27,978 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512014ade625ec8834ba686b91c2b17bf13b6 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512014ade625ec8834ba686b91c2b17bf13b6 2023-03-22 15:54:27,979 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205cd439f363d4940a78da96f6fcc02f817 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205cd439f363d4940a78da96f6fcc02f817 2023-03-22 15:54:27,979 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-22 15:54:27,979 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-22 15:54:27,980 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-22 15:54:27,980 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-22 15:54:27,981 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:27,982 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:54:27,984 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.1070 sec 2023-03-22 15:54:27,993 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): 655cd9582db7f9e343db22d211baf616#family1#compaction#49 average throughput is 14.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:54:28,125 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=268 (was 252) - Thread LEAK? -, OpenFileDescriptor=644 (was 614) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=434 (was 446), ProcessCount=170 (was 170), AvailableMemoryMB=6617 (was 6663) 2023-03-22 15:54:28,141 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=268, OpenFileDescriptor=644, MaxFileDescriptor=60000, SystemLoadAverage=434, ProcessCount=170, AvailableMemoryMB=6616 2023-03-22 15:54:28,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-22 15:54:28,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:28,147 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:54:28,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-22 15:54:28,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-22 15:54:28,149 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:54:28,150 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:28,151 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9 empty. 2023-03-22 15:54:28,151 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:28,152 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-22 15:54:28,162 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-22 15:54:28,163 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => d5f4b1397e7e4f07a82dabc014dab6a9, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:28,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-22 15:54:28,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-22 15:54:28,575 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:28,575 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing d5f4b1397e7e4f07a82dabc014dab6a9, disabling compactions & flushes 2023-03-22 15:54:28,575 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:28,576 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:28,576 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. after waiting 0 ms 2023-03-22 15:54:28,576 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:28,576 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:28,576 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:28,578 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:54:28,579 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500468579"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500468579"}]},"ts":"1679500468579"} 2023-03-22 15:54:28,581 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-22 15:54:28,581 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:54:28,582 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500468582"}]},"ts":"1679500468582"} 2023-03-22 15:54:28,583 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-22 15:54:28,587 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, ASSIGN}] 2023-03-22 15:54:28,588 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, ASSIGN 2023-03-22 15:54:28,589 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:28,741 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:28,741 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500468740"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500468740"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500468740"}]},"ts":"1679500468740"} 2023-03-22 15:54:28,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:28,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-22 15:54:28,842 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/.tmp/family1/b55f6acbddbd40bcb881ea7c8d46e7b1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/b55f6acbddbd40bcb881ea7c8d46e7b1 2023-03-22 15:54:28,848 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 655cd9582db7f9e343db22d211baf616/family1 of 655cd9582db7f9e343db22d211baf616 into b55f6acbddbd40bcb881ea7c8d46e7b1(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:28,848 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:54:28,848 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1679500467970; duration=0sec 2023-03-22 15:54:28,848 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:28,894 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/f1f034c4f0304175994f783c1e756af7, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43b4bdd19a8947e89a436c8d7e71450b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/0aae96f8e4e24525a4a233c31e9a1136_SeqId_47_] to archive 2023-03-22 15:54:28,895 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:28,897 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/f1f034c4f0304175994f783c1e756af7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/f1f034c4f0304175994f783c1e756af7 2023-03-22 15:54:28,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:28,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d5f4b1397e7e4f07a82dabc014dab6a9, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:54:28,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:28,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:28,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:28,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:28,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43b4bdd19a8947e89a436c8d7e71450b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/43b4bdd19a8947e89a436c8d7e71450b 2023-03-22 15:54:28,899 INFO [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:28,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/0aae96f8e4e24525a4a233c31e9a1136_SeqId_47_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/family1/0aae96f8e4e24525a4a233c31e9a1136_SeqId_47_ 2023-03-22 15:54:28,901 DEBUG [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:28,901 DEBUG [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:28,901 INFO [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d5f4b1397e7e4f07a82dabc014dab6a9 columnFamilyName family1 2023-03-22 15:54:28,903 INFO [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] regionserver.HStore(310): Store=d5f4b1397e7e4f07a82dabc014dab6a9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:28,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:28,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:28,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:28,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:28,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d5f4b1397e7e4f07a82dabc014dab6a9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61681821, jitterRate=-0.08086924254894257}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:54:28,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:28,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9., pid=41, masterSystemTime=1679500468895 2023-03-22 15:54:28,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:28,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:28,912 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:28,912 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500468912"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500468912"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500468912"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500468912"}]},"ts":"1679500468912"} 2023-03-22 15:54:28,915 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-22 15:54:28,915 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185 in 170 msec 2023-03-22 15:54:28,917 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-22 15:54:28,917 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, ASSIGN in 329 msec 2023-03-22 15:54:28,918 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:54:28,918 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500468918"}]},"ts":"1679500468918"} 2023-03-22 15:54:28,919 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-22 15:54:28,923 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:54:28,930 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 778 msec 2023-03-22 15:54:29,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-22 15:54:29,253 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-22 15:54:29,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:29,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,286 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:29,286 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:29,286 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:29,286 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:29,287 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,287 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,288 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,288 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:29,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:29,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:29,290 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,290 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:29,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:29,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,291 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,291 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,291 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:29,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:29,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:29,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:29,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:29,292 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:29,292 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:29,293 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-22 15:54:29,341 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130246ac8a5ce314945b4d693c4353589d2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130246ac8a5ce314945b4d693c4353589d2 2023-03-22 15:54:29,347 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/ecd0f1ca5a8e4b3d805a7d26ae9cbdd8 2023-03-22 15:54:29,353 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/ecd0f1ca5a8e4b3d805a7d26ae9cbdd8 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/ecd0f1ca5a8e4b3d805a7d26ae9cbdd8 2023-03-22 15:54:29,358 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/ecd0f1ca5a8e4b3d805a7d26ae9cbdd8, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-22 15:54:29,360 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 67ms, sequenceid=5, compaction requested=false 2023-03-22 15:54:29,360 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:29,360 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:29,361 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:29,361 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:29,361 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:29,361 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,361 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:29,361 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:29,362 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:29,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:29,363 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:29,363 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,363 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:29,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:29,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:29,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:29,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ceeb859[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:29,365 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:29,365 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,367 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,367 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,367 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:29,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,372 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:29,372 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:29,372 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:29,375 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:29,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:29,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:29,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:29,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:29,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:29,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:29,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:29,378 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:29,379 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:29,379 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:29,379 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,379 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:29,381 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,381 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:29,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,381 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:29,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:29,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:29,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:29,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:29,382 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:29,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,383 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:29,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,386 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,386 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:29,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:29,386 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:29,386 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,386 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:29,387 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:29,387 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:29,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:29,386 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:29,387 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,387 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:29,387 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,387 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:29,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:29,487 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:29,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:29,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:29,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:29,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,495 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:29,495 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:29,496 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:29,496 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:29,496 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,496 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,498 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,498 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:29,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:29,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:29,498 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:29,498 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:29,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:29,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-22 15:54:29,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:29,499 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:29,500 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:29,500 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:29,500 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-22 15:54:29,914 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128d4eedfd4ad4244e69dd429f6c2412cc4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128d4eedfd4ad4244e69dd429f6c2412cc4 2023-03-22 15:54:30,320 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/1dca2141301f40789309c196e544c88b 2023-03-22 15:54:30,328 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/1dca2141301f40789309c196e544c88b as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/1dca2141301f40789309c196e544c88b 2023-03-22 15:54:30,334 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/1dca2141301f40789309c196e544c88b, entries=1, sequenceid=9, filesize=5.0 K 2023-03-22 15:54:30,335 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 835ms, sequenceid=9, compaction requested=false 2023-03-22 15:54:30,336 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:30,336 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:30,336 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:30,336 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:30,336 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,336 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:30,336 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:30,339 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:30,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:30,339 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:30,339 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,339 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:30,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:30,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:30,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:30,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41ed4be3[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:30,341 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:30,341 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,342 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,343 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:30,343 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:30,343 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:30,343 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,343 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:30,344 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,344 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:30,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:30,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:30,344 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:30,345 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:30,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:30,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:30,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:30,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:30,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:30,348 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:30,348 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:30,348 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:30,348 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,348 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:30,349 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,350 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,350 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:30,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:30,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:30,350 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:30,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:30,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:30,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,352 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:30,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:30,358 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,358 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,358 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:30,358 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,358 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,358 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,358 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:30,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:30,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:30,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:30,358 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,358 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:30,358 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:30,359 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:30,359 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:30,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:30,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:30,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-22 15:54:30,459 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:30,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:30,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:30,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:30,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,476 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:30,476 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:30,476 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:30,476 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:30,477 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,477 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,478 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,478 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:30,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:30,479 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,479 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:30,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,480 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,484 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-22 15:54:30,484 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,484 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:30,484 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:30,484 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:30,484 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:30,484 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:30,484 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:30,484 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:30,485 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:30,931 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120112adecc00c7040169a07a8cbc6ccf3b4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120112adecc00c7040169a07a8cbc6ccf3b4 2023-03-22 15:54:30,942 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/55cf6c692ede4b10a51a6ba285098a39 2023-03-22 15:54:30,949 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/55cf6c692ede4b10a51a6ba285098a39 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/55cf6c692ede4b10a51a6ba285098a39 2023-03-22 15:54:30,954 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/55cf6c692ede4b10a51a6ba285098a39, entries=1, sequenceid=13, filesize=5.0 K 2023-03-22 15:54:30,956 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 471ms, sequenceid=13, compaction requested=true 2023-03-22 15:54:30,957 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:30,957 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:30,957 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:30,957 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:30,957 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,957 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:30,957 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:30,959 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,959 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:30,959 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,959 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e565b25[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:30,962 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:30,962 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,964 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,964 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:30,964 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:30,964 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:30,964 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,964 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:30,967 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:30,967 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:30,967 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:30,967 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:30,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:30,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:30,972 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:30,972 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:30,972 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:30,972 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:30,972 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,972 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:30,974 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,974 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:30,974 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:30,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,980 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,982 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,984 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:30,984 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:30,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:30,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:30,985 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,985 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,985 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-22 15:54:30,985 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:30,985 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,985 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,985 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:30,985 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:30,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:30,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:30,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:30,985 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:30,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:30,986 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:31,086 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:31,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:31,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:31,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:31,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,094 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:31,094 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:31,094 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:31,094 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:31,094 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,094 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,096 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,096 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:31,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:31,096 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,096 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,096 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:31,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,097 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,097 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-22 15:54:31,097 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,097 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:31,097 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:31,097 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:31,104 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:31,104 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:31,104 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:31,531 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512059057a48fa5e248cd93e0ff9b83b8100c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512059057a48fa5e248cd93e0ff9b83b8100c 2023-03-22 15:54:31,537 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/98725b71c5d14b14aa1e7481c35dc6e5 2023-03-22 15:54:31,541 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/98725b71c5d14b14aa1e7481c35dc6e5 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/98725b71c5d14b14aa1e7481c35dc6e5 2023-03-22 15:54:31,546 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/98725b71c5d14b14aa1e7481c35dc6e5, entries=1, sequenceid=17, filesize=5.0 K 2023-03-22 15:54:31,547 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 443ms, sequenceid=17, compaction requested=true 2023-03-22 15:54:31,547 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:31,547 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:31,547 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:31,547 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:31,547 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,547 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:31,547 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:31,550 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,550 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:31,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:31,550 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,550 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:31,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:31,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:31,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:31,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:31,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b609b49[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:31,552 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:31,552 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,553 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,553 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:31,553 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:31,553 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:31,553 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,553 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:31,555 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,555 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:31,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:31,555 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:31,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:31,555 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:31,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:31,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:31,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:31,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:31,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:31,557 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:31,558 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:31,558 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:31,558 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,558 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:31,559 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,559 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:31,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:31,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,559 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:31,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:31,560 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:31,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:31,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:31,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,561 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:31,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,564 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,564 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,564 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:31,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:31,564 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:31,564 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:31,564 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:31,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:31,565 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:31,565 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,565 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:31,565 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:31,565 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:31,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:31,565 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,665 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:31,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:31,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:31,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:31,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,673 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:31,673 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:31,673 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:31,673 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:31,679 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,679 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,681 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,681 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:31,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:31,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:31,682 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:31,682 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:31,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,682 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,683 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-22 15:54:31,683 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:31,683 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:31,683 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:31,683 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:31,688 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:31,688 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:31,688 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:32,112 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512285bc7fdc9b10245748085c9747d7a3711 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512285bc7fdc9b10245748085c9747d7a3711 2023-03-22 15:54:32,518 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/d22d10892fb042ab95449954c2f1ece1 2023-03-22 15:54:32,525 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/d22d10892fb042ab95449954c2f1ece1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/d22d10892fb042ab95449954c2f1ece1 2023-03-22 15:54:32,530 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/d22d10892fb042ab95449954c2f1ece1, entries=1, sequenceid=21, filesize=5.0 K 2023-03-22 15:54:32,531 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 843ms, sequenceid=21, compaction requested=true 2023-03-22 15:54:32,531 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:32,531 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:32,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:32,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:32,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:32,531 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:32,533 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,533 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:32,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:32,534 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,534 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:32,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:32,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:32,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:32,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:32,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@265d20c[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:32,535 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:32,535 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,539 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:32,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:32,540 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,540 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:32,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:32,541 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,541 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:32,542 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:32,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,542 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:32,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:32,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:32,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:32,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:32,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:32,545 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:32,545 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:32,545 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:32,545 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,545 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:32,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:32,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:32,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:32,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,547 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:32,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:32,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,550 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:32,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:32,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:32,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-22 15:54:32,553 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:32,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,552 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:32,553 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,553 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:32,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:32,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:32,553 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:32,553 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:32,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:32,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:32,653 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:32,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:32,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:32,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:32,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,660 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:32,660 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:32,660 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:32,660 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:32,661 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,661 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,663 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:32,663 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:32,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:32,663 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:32,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,663 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:32,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:32,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,664 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,664 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-22 15:54:32,664 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:32,664 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:32,665 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:32,665 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:32,665 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:32,665 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:32,665 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:32,665 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:32,665 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:32,682 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231719bb78b186e437386dd7dd6a0161826 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231719bb78b186e437386dd7dd6a0161826 2023-03-22 15:54:33,095 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/2a952dac32f544bd99b5ee7a8c78ea56 2023-03-22 15:54:33,099 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/2a952dac32f544bd99b5ee7a8c78ea56 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/2a952dac32f544bd99b5ee7a8c78ea56 2023-03-22 15:54:33,103 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/2a952dac32f544bd99b5ee7a8c78ea56, entries=1, sequenceid=25, filesize=5.0 K 2023-03-22 15:54:33,104 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 439ms, sequenceid=25, compaction requested=true 2023-03-22 15:54:33,104 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:33,104 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:33,104 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:33,104 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:33,104 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,104 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:33,104 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:33,106 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,106 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:33,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:33,106 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,106 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:33,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:33,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:33,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:33,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:33,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@297c23f0[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:33,108 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:33,108 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,109 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,109 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:33,109 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:33,109 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:33,109 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,109 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:33,110 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,110 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:33,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:33,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:33,111 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:33,111 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:33,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:33,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:33,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:33,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:33,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:33,113 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:33,113 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:33,113 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:33,113 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,113 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:33,115 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,115 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:33,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:33,116 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:33,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:33,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:33,116 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:33,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:33,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,117 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:33,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:33,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,119 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:33,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:33,119 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:33,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:33,220 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:33,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:33,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:33,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:33,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,226 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:33,226 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:33,226 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:33,226 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:33,226 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,226 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,228 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,228 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:33,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:33,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:33,228 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:33,228 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:33,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:33,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-22 15:54:33,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:33,229 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:33,229 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:33,229 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:33,229 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:33,229 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:33,229 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:33,229 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:33,230 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:33,642 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010170ee7c44d2a546088d73560188c6f60e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010170ee7c44d2a546088d73560188c6f60e 2023-03-22 15:54:34,047 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/10acbbaf3152482495f030fe7c65de6f 2023-03-22 15:54:34,053 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/10acbbaf3152482495f030fe7c65de6f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/10acbbaf3152482495f030fe7c65de6f 2023-03-22 15:54:34,057 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/10acbbaf3152482495f030fe7c65de6f, entries=1, sequenceid=29, filesize=5.0 K 2023-03-22 15:54:34,058 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 828ms, sequenceid=29, compaction requested=true 2023-03-22 15:54:34,058 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:34,058 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:34,058 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:34,058 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:34,058 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,058 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:34,058 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:34,060 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,060 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:34,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:34,060 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,060 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:34,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:34,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:34,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:34,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:34,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@550fda89[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:34,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:34,062 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,063 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,063 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:34,063 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,063 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:34,063 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:34,063 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:34,064 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,064 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:34,064 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:34,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:34,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:34,064 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:34,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:34,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:34,066 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:34,066 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:34,066 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:34,066 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,066 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:34,069 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,069 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,069 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:34,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:34,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:34,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:34,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:34,070 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:34,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:34,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,070 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:34,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-22 15:54:34,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:34,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,075 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:34,075 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,076 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:34,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:34,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:34,176 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:34,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:34,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:34,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:34,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,182 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:34,182 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:34,182 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:34,182 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:34,182 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,182 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,184 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,184 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:34,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:34,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:34,184 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:34,184 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:34,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,184 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:34,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,185 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,185 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-22 15:54:34,185 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:34,185 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:34,185 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:34,185 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:34,427 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:54:34,597 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b01d952687ee4c158fe48146b0938c12 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b01d952687ee4c158fe48146b0938c12 2023-03-22 15:54:35,003 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/aa6598e2ee934bac90f152ed1fc5589b 2023-03-22 15:54:35,007 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/aa6598e2ee934bac90f152ed1fc5589b as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/aa6598e2ee934bac90f152ed1fc5589b 2023-03-22 15:54:35,011 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/aa6598e2ee934bac90f152ed1fc5589b, entries=1, sequenceid=33, filesize=5.0 K 2023-03-22 15:54:35,012 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 827ms, sequenceid=33, compaction requested=true 2023-03-22 15:54:35,012 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:35,012 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:35,012 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:35,012 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:35,012 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,012 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:35,012 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:35,014 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,014 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:35,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:35,015 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,015 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:35,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:35,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:35,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:35,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:35,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a1aa9b2[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:35,016 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:35,016 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,017 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,017 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:35,017 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:35,017 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:35,017 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,017 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:35,019 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,019 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:35,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:35,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:35,019 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:35,019 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:35,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:35,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:35,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:35,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:35,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:35,021 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:35,021 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:35,021 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:35,021 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,021 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:35,023 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,023 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:35,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,023 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:35,023 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,024 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:35,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,027 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,027 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:35,027 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,027 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:35,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:35,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:35,027 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,027 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,027 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:35,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:35,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-22 15:54:35,027 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,028 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:35,028 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:35,028 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:35,028 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:35,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:35,128 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:35,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:35,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:35,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:35,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,134 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:35,134 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:35,135 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:35,135 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:35,135 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,135 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,136 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:35,136 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:35,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:35,137 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,137 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:35,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:35,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-22 15:54:35,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,138 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:35,138 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:35,138 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:35,138 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:35,138 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:35,138 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:35,138 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:35,138 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:35,551 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907317079c4cbed4f50a9dbcacc7c72b6cf to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907317079c4cbed4f50a9dbcacc7c72b6cf 2023-03-22 15:54:35,956 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/b634034db0b7482ea71bf9b054b5b807 2023-03-22 15:54:35,960 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/b634034db0b7482ea71bf9b054b5b807 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b634034db0b7482ea71bf9b054b5b807 2023-03-22 15:54:35,964 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b634034db0b7482ea71bf9b054b5b807, entries=1, sequenceid=37, filesize=5.0 K 2023-03-22 15:54:35,965 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 827ms, sequenceid=37, compaction requested=true 2023-03-22 15:54:35,965 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:35,965 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:35,965 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:35,965 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:35,965 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,965 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:35,965 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:35,967 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,967 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:35,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:35,967 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,967 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:35,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:35,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:35,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:35,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:35,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5643e14f[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:35,968 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:35,968 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,969 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,970 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:35,970 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:35,970 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:35,970 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,970 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:35,974 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:35,976 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:35,976 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:35,977 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:35,977 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,977 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:35,978 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,978 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,978 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:35,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:35,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:35,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:35,979 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:35,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:35,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:35,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,980 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:35,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:35,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:35,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:35,982 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-22 15:54:35,982 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:35,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:35,982 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:35,983 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,082 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:36,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:36,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:36,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:36,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,089 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:36,089 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:36,089 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-22 15:54:36,089 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-22 15:54:36,089 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,090 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,096 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:36,096 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:36,096 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,096 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,097 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:36,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,097 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,097 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-22 15:54:36,097 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,097 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-22 15:54:36,097 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:36,097 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-22 15:54:36,098 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:36,098 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:36,098 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:36,098 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. started... 2023-03-22 15:54:36,098 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d5f4b1397e7e4f07a82dabc014dab6a9 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:36,510 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511204fbc73c1c411490983168b62c9245997 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511204fbc73c1c411490983168b62c9245997 2023-03-22 15:54:36,915 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/29fd0777a25e46508fece68871feaf5f 2023-03-22 15:54:36,921 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/29fd0777a25e46508fece68871feaf5f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/29fd0777a25e46508fece68871feaf5f 2023-03-22 15:54:36,925 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/29fd0777a25e46508fece68871feaf5f, entries=1, sequenceid=41, filesize=5.0 K 2023-03-22 15:54:36,925 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d5f4b1397e7e4f07a82dabc014dab6a9 in 827ms, sequenceid=41, compaction requested=true 2023-03-22 15:54:36,925 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:36,925 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:36,926 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:36,926 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:36,926 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,926 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-22 15:54:36,926 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-22 15:54:36,928 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,928 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:36,928 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,928 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:36,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:36,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-22 15:54:36,929 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-22 15:54:36,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ee83bd4[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:36,929 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,930 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,931 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-22 15:54:36,931 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-22 15:54:36,931 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,931 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:36,931 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:36,932 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,932 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-22 15:54:36,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:36,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:36,932 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:36,932 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-22 15:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:36,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:36,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-22 15:54:36,934 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-22 15:54:36,934 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:36,934 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:36,934 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,934 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:36,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:36,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:36,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:36,937 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,938 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,940 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,940 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:36,940 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-22 15:54:36,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:36,940 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,940 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:36,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:36,940 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:36,940 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:36,941 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:36,941 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:36,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:36,941 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,941 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:36,941 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:37,041 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:37,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:37,043 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:37,044 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-22 15:54:37,045 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-22 15:54:37,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:37,046 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501077045 2023-03-22 15:54:37,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:54:37,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-22 15:54:37,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:37,049 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x360e9f25 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:37,053 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@119a8e89, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:37,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:37,888 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030278f94995dd491496267897bda70c00 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601030278f94995dd491496267897bda70c00 2023-03-22 15:54:37,889 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:37,901 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:54:37,902 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58308, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:54:37,907 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:54:37,909 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45430, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:54:37,918 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6cc1c6879e234d948202384b7677d5b1 first=Optional[r3] last=Optional[r6] 2023-03-22 15:54:37,919 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6cc1c6879e234d948202384b7677d5b1}] 2023-03-22 15:54:37,924 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6cc1c6879e234d948202384b7677d5b1 for inclusion in d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:37,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-22 15:54:37,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:37,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:37,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6cc1c6879e234d948202384b7677d5b1 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anj7u4285p7l6idcrgchgpkq5sllkfip5pmaic0i3uaanojs8s91u7mmb1ln5p6p/family1/6cc1c6879e234d948202384b7677d5b1 2023-03-22 15:54:37,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anj7u4285p7l6idcrgchgpkq5sllkfip5pmaic0i3uaanojs8s91u7mmb1ln5p6p/family1/6cc1c6879e234d948202384b7677d5b1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/46968cccb35442768bfbe70b3b697558_SeqId_44_ 2023-03-22 15:54:37,930 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anj7u4285p7l6idcrgchgpkq5sllkfip5pmaic0i3uaanojs8s91u7mmb1ln5p6p/family1/6cc1c6879e234d948202384b7677d5b1 into d5f4b1397e7e4f07a82dabc014dab6a9/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/46968cccb35442768bfbe70b3b697558_SeqId_44_ - updating store file list. 2023-03-22 15:54:37,933 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/46968cccb35442768bfbe70b3b697558_SeqId_44_ into d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:37,933 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anj7u4285p7l6idcrgchgpkq5sllkfip5pmaic0i3uaanojs8s91u7mmb1ln5p6p/family1/6cc1c6879e234d948202384b7677d5b1 into d5f4b1397e7e4f07a82dabc014dab6a9/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/46968cccb35442768bfbe70b3b697558_SeqId_44_) 2023-03-22 15:54:37,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anj7u4285p7l6idcrgchgpkq5sllkfip5pmaic0i3uaanojs8s91u7mmb1ln5p6p/family1/6cc1c6879e234d948202384b7677d5b1 2023-03-22 15:54:37,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-22 15:54:37,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-22 15:54:37,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): d5f4b1397e7e4f07a82dabc014dab6a9/family1 is initiating minor compaction 2023-03-22 15:54:37,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6b515044; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-22 15:54:37,937 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d5f4b1397e7e4f07a82dabc014dab6a9/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:37,937 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/1dca2141301f40789309c196e544c88b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/55cf6c692ede4b10a51a6ba285098a39, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/98725b71c5d14b14aa1e7481c35dc6e5, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/d22d10892fb042ab95449954c2f1ece1, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/2a952dac32f544bd99b5ee7a8c78ea56, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/10acbbaf3152482495f030fe7c65de6f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/aa6598e2ee934bac90f152ed1fc5589b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b634034db0b7482ea71bf9b054b5b807, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/29fd0777a25e46508fece68871feaf5f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/46968cccb35442768bfbe70b3b697558_SeqId_44_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp, totalSize=50.4 K 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 1dca2141301f40789309c196e544c88b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 55cf6c692ede4b10a51a6ba285098a39, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 98725b71c5d14b14aa1e7481c35dc6e5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting d22d10892fb042ab95449954c2f1ece1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 2a952dac32f544bd99b5ee7a8c78ea56, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 10acbbaf3152482495f030fe7c65de6f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting aa6598e2ee934bac90f152ed1fc5589b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting b634034db0b7482ea71bf9b054b5b807, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 29fd0777a25e46508fece68871feaf5f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-22 15:54:37,937 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 46968cccb35442768bfbe70b3b697558_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-22 15:54:37,938 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:37,939 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:37,941 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512285bc7fdc9b10245748085c9747d7a3711 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512285bc7fdc9b10245748085c9747d7a3711 2023-03-22 15:54:37,942 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231719bb78b186e437386dd7dd6a0161826 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231719bb78b186e437386dd7dd6a0161826 2023-03-22 15:54:37,943 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010170ee7c44d2a546088d73560188c6f60e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010170ee7c44d2a546088d73560188c6f60e 2023-03-22 15:54:37,944 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b01d952687ee4c158fe48146b0938c12 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b01d952687ee4c158fe48146b0938c12 2023-03-22 15:54:37,944 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-22 15:54:37,944 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-22 15:54:37,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:37,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:37,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:37,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:37,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:37,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:37,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:37,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:37,965 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): d5f4b1397e7e4f07a82dabc014dab6a9#family1#compaction#60 average throughput is 1.59 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:54:38,601 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:54:38,601 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6a3434be to 127.0.0.1:58225 2023-03-22 15:54:38,601 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:54:38,601 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:54:38,601 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2cc37329 to 127.0.0.1:58225 2023-03-22 15:54:38,601 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:54:38,601 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:54:38,601 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x63c134eb to 127.0.0.1:58225 2023-03-22 15:54:38,601 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:54:38,788 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/b6de95d86a2a4a3a9ea2811d6ee758b0 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b6de95d86a2a4a3a9ea2811d6ee758b0 2023-03-22 15:54:38,789 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120574c32cc8e2314c5dba89a40a4f504e47 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120574c32cc8e2314c5dba89a40a4f504e47 2023-03-22 15:54:38,790 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:38,793 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d5f4b1397e7e4f07a82dabc014dab6a9/family1 of d5f4b1397e7e4f07a82dabc014dab6a9 into b6de95d86a2a4a3a9ea2811d6ee758b0(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:38,793 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:38,793 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1679500477937; duration=0sec 2023-03-22 15:54:38,793 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:38,813 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d75f41e829d3472a8f582d9c29855332 first=Optional[r00] last=Optional[r2] 2023-03-22 15:54:38,814 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d75f41e829d3472a8f582d9c29855332}] 2023-03-22 15:54:38,815 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d75f41e829d3472a8f582d9c29855332 for inclusion in d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:38,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-22 15:54:38,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:38,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:38,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d75f41e829d3472a8f582d9c29855332 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__m69ckcfadsmnnjehjav524k537q4locklh7ajbuj6ktfq8qfassarcrbt1k1hn24/family1/d75f41e829d3472a8f582d9c29855332 2023-03-22 15:54:38,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__m69ckcfadsmnnjehjav524k537q4locklh7ajbuj6ktfq8qfassarcrbt1k1hn24/family1/d75f41e829d3472a8f582d9c29855332 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/709d6ce0e3ba42ecb21ae09fce82879a_SeqId_47_ 2023-03-22 15:54:38,821 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__m69ckcfadsmnnjehjav524k537q4locklh7ajbuj6ktfq8qfassarcrbt1k1hn24/family1/d75f41e829d3472a8f582d9c29855332 into d5f4b1397e7e4f07a82dabc014dab6a9/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/709d6ce0e3ba42ecb21ae09fce82879a_SeqId_47_ - updating store file list. 2023-03-22 15:54:38,824 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/709d6ce0e3ba42ecb21ae09fce82879a_SeqId_47_ into d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:38,824 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__m69ckcfadsmnnjehjav524k537q4locklh7ajbuj6ktfq8qfassarcrbt1k1hn24/family1/d75f41e829d3472a8f582d9c29855332 into d5f4b1397e7e4f07a82dabc014dab6a9/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/709d6ce0e3ba42ecb21ae09fce82879a_SeqId_47_) 2023-03-22 15:54:38,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__m69ckcfadsmnnjehjav524k537q4locklh7ajbuj6ktfq8qfassarcrbt1k1hn24/family1/d75f41e829d3472a8f582d9c29855332 2023-03-22 15:54:38,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-22 15:54:38,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-22 15:54:38,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): d5f4b1397e7e4f07a82dabc014dab6a9/family1 is initiating minor compaction (all files) 2023-03-22 15:54:38,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2bd13587; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:38,826 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d5f4b1397e7e4f07a82dabc014dab6a9/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:38,826 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/ecd0f1ca5a8e4b3d805a7d26ae9cbdd8, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b6de95d86a2a4a3a9ea2811d6ee758b0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/709d6ce0e3ba42ecb21ae09fce82879a_SeqId_47_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp, totalSize=390.1 K 2023-03-22 15:54:38,826 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting ecd0f1ca5a8e4b3d805a7d26ae9cbdd8, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840482 2023-03-22 15:54:38,827 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting b6de95d86a2a4a3a9ea2811d6ee758b0, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240482 2023-03-22 15:54:38,827 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 709d6ce0e3ba42ecb21ae09fce82879a_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840482 2023-03-22 15:54:38,828 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:38,828 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:38,830 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130246ac8a5ce314945b4d693c4353589d2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130246ac8a5ce314945b4d693c4353589d2 2023-03-22 15:54:38,831 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120112adecc00c7040169a07a8cbc6ccf3b4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120112adecc00c7040169a07a8cbc6ccf3b4 2023-03-22 15:54:38,832 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512059057a48fa5e248cd93e0ff9b83b8100c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512059057a48fa5e248cd93e0ff9b83b8100c 2023-03-22 15:54:38,833 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-22 15:54:38,833 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-22 15:54:38,833 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:38,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:38,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:38,835 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-22 15:54:38,836 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.7920 sec 2023-03-22 15:54:38,847 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): d5f4b1397e7e4f07a82dabc014dab6a9#family1#compaction#61 average throughput is 30.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:54:38,896 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/1dca2141301f40789309c196e544c88b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/55cf6c692ede4b10a51a6ba285098a39, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/98725b71c5d14b14aa1e7481c35dc6e5, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/d22d10892fb042ab95449954c2f1ece1, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/2a952dac32f544bd99b5ee7a8c78ea56, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/10acbbaf3152482495f030fe7c65de6f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/aa6598e2ee934bac90f152ed1fc5589b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b634034db0b7482ea71bf9b054b5b807, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/29fd0777a25e46508fece68871feaf5f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/46968cccb35442768bfbe70b3b697558_SeqId_44_] to archive 2023-03-22 15:54:38,896 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:38,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-22 15:54:38,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/1dca2141301f40789309c196e544c88b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/1dca2141301f40789309c196e544c88b 2023-03-22 15:54:38,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:38,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/55cf6c692ede4b10a51a6ba285098a39 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/55cf6c692ede4b10a51a6ba285098a39 2023-03-22 15:54:38,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/98725b71c5d14b14aa1e7481c35dc6e5 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/98725b71c5d14b14aa1e7481c35dc6e5 2023-03-22 15:54:38,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/d22d10892fb042ab95449954c2f1ece1 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/d22d10892fb042ab95449954c2f1ece1 2023-03-22 15:54:38,903 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/2a952dac32f544bd99b5ee7a8c78ea56 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/2a952dac32f544bd99b5ee7a8c78ea56 2023-03-22 15:54:38,904 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/10acbbaf3152482495f030fe7c65de6f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/10acbbaf3152482495f030fe7c65de6f 2023-03-22 15:54:38,905 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/aa6598e2ee934bac90f152ed1fc5589b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/aa6598e2ee934bac90f152ed1fc5589b 2023-03-22 15:54:38,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b634034db0b7482ea71bf9b054b5b807 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b634034db0b7482ea71bf9b054b5b807 2023-03-22 15:54:38,907 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/29fd0777a25e46508fece68871feaf5f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/29fd0777a25e46508fece68871feaf5f 2023-03-22 15:54:38,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/46968cccb35442768bfbe70b3b697558_SeqId_44_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/46968cccb35442768bfbe70b3b697558_SeqId_44_ 2023-03-22 15:54:39,307 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-22 15:54:39,308 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-22 15:54:39,308 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-22 15:54:39,310 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-22 15:54:39,313 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, REOPEN/MOVE}] 2023-03-22 15:54:39,313 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, REOPEN/MOVE 2023-03-22 15:54:39,314 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:39,314 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500479314"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500479314"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500479314"}]},"ts":"1679500479314"} 2023-03-22 15:54:39,315 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:39,467 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:39,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d5f4b1397e7e4f07a82dabc014dab6a9, disabling compactions & flushes 2023-03-22 15:54:39,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:40,070 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/.tmp/family1/13f1a0d9d9ab43ceb16562e9f79afab9 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/13f1a0d9d9ab43ceb16562e9f79afab9 2023-03-22 15:54:40,076 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d5f4b1397e7e4f07a82dabc014dab6a9/family1 of d5f4b1397e7e4f07a82dabc014dab6a9 into 13f1a0d9d9ab43ceb16562e9f79afab9(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:54:40,077 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:40,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:40,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. after waiting 0 ms 2023-03-22 15:54:40,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:40,077 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:40,077 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1679500478826; duration=1sec 2023-03-22 15:54:40,077 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/ecd0f1ca5a8e4b3d805a7d26ae9cbdd8, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b6de95d86a2a4a3a9ea2811d6ee758b0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/709d6ce0e3ba42ecb21ae09fce82879a_SeqId_47_] to archive 2023-03-22 15:54:40,077 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:40,078 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:40,080 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/ecd0f1ca5a8e4b3d805a7d26ae9cbdd8 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/ecd0f1ca5a8e4b3d805a7d26ae9cbdd8 2023-03-22 15:54:40,081 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b6de95d86a2a4a3a9ea2811d6ee758b0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/b6de95d86a2a4a3a9ea2811d6ee758b0 2023-03-22 15:54:40,083 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/709d6ce0e3ba42ecb21ae09fce82879a_SeqId_47_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/709d6ce0e3ba42ecb21ae09fce82879a_SeqId_47_ 2023-03-22 15:54:40,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-22 15:54:40,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:40,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:40,090 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: d5f4b1397e7e4f07a82dabc014dab6a9 to self. 2023-03-22 15:54:40,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:40,092 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=CLOSED 2023-03-22 15:54:40,092 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500480092"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500480092"}]},"ts":"1679500480092"} 2023-03-22 15:54:40,095 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-22 15:54:40,095 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185 in 779 msec 2023-03-22 15:54:40,096 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=true 2023-03-22 15:54:40,247 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:40,247 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500480247"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500480247"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500480247"}]},"ts":"1679500480247"} 2023-03-22 15:54:40,249 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:40,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:40,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d5f4b1397e7e4f07a82dabc014dab6a9, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:54:40,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:40,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:40,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:40,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:40,407 INFO [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:40,408 DEBUG [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:40,408 DEBUG [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:40,409 INFO [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d5f4b1397e7e4f07a82dabc014dab6a9 columnFamilyName family1 2023-03-22 15:54:40,415 DEBUG [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/13f1a0d9d9ab43ceb16562e9f79afab9 2023-03-22 15:54:40,415 INFO [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] regionserver.HStore(310): Store=d5f4b1397e7e4f07a82dabc014dab6a9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:40,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:40,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:40,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:40,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d5f4b1397e7e4f07a82dabc014dab6a9; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61011593, jitterRate=-0.09085641801357269}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:54:40,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:40,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9., pid=47, masterSystemTime=1679500480401 2023-03-22 15:54:40,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:40,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:40,426 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:40,426 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500480426"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500480426"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500480426"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500480426"}]},"ts":"1679500480426"} 2023-03-22 15:54:40,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-22 15:54:40,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185 in 179 msec 2023-03-22 15:54:40,436 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-22 15:54:40,436 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, REOPEN/MOVE in 1.1160 sec 2023-03-22 15:54:40,439 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-22 15:54:40,439 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1280 sec 2023-03-22 15:54:40,446 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5470 sec 2023-03-22 15:54:40,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-22 15:54:40,449 INFO [Listener at localhost/43289] compactions.TestMobCompactor(278): alter status finished 2023-03-22 15:54:40,450 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:40,452 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-22 15:54:40,453 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-22 15:54:40,454 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501080454 2023-03-22 15:54:40,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:40,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:54:40,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-22 15:54:40,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:40,457 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x3661ce42 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:40,493 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@734af0c1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:40,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:40,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:40,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:40,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:40,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:40,637 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511285164e31adcbd46c588886cb1328d8c99 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511285164e31adcbd46c588886cb1328d8c99 2023-03-22 15:54:40,640 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:40,680 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:54:40,681 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58312, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:54:40,713 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:54:40,716 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45436, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:54:40,753 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b846acf2166f4435998a16dd0143bc26 first=Optional[r05] last=Optional[r8] 2023-03-22 15:54:40,754 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b846acf2166f4435998a16dd0143bc26}] 2023-03-22 15:54:40,776 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b846acf2166f4435998a16dd0143bc26 for inclusion in d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:40,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-22 15:54:40,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:40,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:40,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b846acf2166f4435998a16dd0143bc26 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ap0tu8bis03ue78p1ee5mf3s88m16u05mlqmmp4qr1dkmgertmrlls500a8bkju2/family1/b846acf2166f4435998a16dd0143bc26 2023-03-22 15:54:40,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ap0tu8bis03ue78p1ee5mf3s88m16u05mlqmmp4qr1dkmgertmrlls500a8bkju2/family1/b846acf2166f4435998a16dd0143bc26 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/27f0c2ace28a45b4a1ca97716314b03d_SeqId_53_ 2023-03-22 15:54:40,793 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ap0tu8bis03ue78p1ee5mf3s88m16u05mlqmmp4qr1dkmgertmrlls500a8bkju2/family1/b846acf2166f4435998a16dd0143bc26 into d5f4b1397e7e4f07a82dabc014dab6a9/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/27f0c2ace28a45b4a1ca97716314b03d_SeqId_53_ - updating store file list. 2023-03-22 15:54:40,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/27f0c2ace28a45b4a1ca97716314b03d_SeqId_53_ into d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:40,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ap0tu8bis03ue78p1ee5mf3s88m16u05mlqmmp4qr1dkmgertmrlls500a8bkju2/family1/b846acf2166f4435998a16dd0143bc26 into d5f4b1397e7e4f07a82dabc014dab6a9/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/27f0c2ace28a45b4a1ca97716314b03d_SeqId_53_) 2023-03-22 15:54:40,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__ap0tu8bis03ue78p1ee5mf3s88m16u05mlqmmp4qr1dkmgertmrlls500a8bkju2/family1/b846acf2166f4435998a16dd0143bc26 2023-03-22 15:54:40,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:40,805 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:40,807 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511204fbc73c1c411490983168b62c9245997 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511204fbc73c1c411490983168b62c9245997 2023-03-22 15:54:40,809 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128d4eedfd4ad4244e69dd429f6c2412cc4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128d4eedfd4ad4244e69dd429f6c2412cc4 2023-03-22 15:54:40,809 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-22 15:54:40,809 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-22 15:54:40,810 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:40,810 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:40,810 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:40,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:40,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:40,812 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-22 15:54:40,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 361 msec 2023-03-22 15:54:40,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-22 15:54:40,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:41,345 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-22 15:54:41,346 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-22 15:54:41,346 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-22 15:54:41,349 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-22 15:54:41,352 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, REOPEN/MOVE}] 2023-03-22 15:54:41,353 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, REOPEN/MOVE 2023-03-22 15:54:41,354 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:41,354 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500481354"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500481354"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500481354"}]},"ts":"1679500481354"} 2023-03-22 15:54:41,355 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:41,507 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:41,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d5f4b1397e7e4f07a82dabc014dab6a9, disabling compactions & flushes 2023-03-22 15:54:41,508 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:41,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:41,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. after waiting 0 ms 2023-03-22 15:54:41,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:41,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-22 15:54:41,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:41,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:41,516 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: d5f4b1397e7e4f07a82dabc014dab6a9 to self. 2023-03-22 15:54:41,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:41,518 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=CLOSED 2023-03-22 15:54:41,518 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500481518"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500481518"}]},"ts":"1679500481518"} 2023-03-22 15:54:41,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-22 15:54:41,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185 in 165 msec 2023-03-22 15:54:41,522 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=true 2023-03-22 15:54:41,673 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:41,673 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500481673"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500481673"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500481673"}]},"ts":"1679500481673"} 2023-03-22 15:54:41,677 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:41,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:41,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d5f4b1397e7e4f07a82dabc014dab6a9, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:54:41,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:41,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:41,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:41,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:41,835 INFO [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:41,836 DEBUG [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:41,836 DEBUG [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1 2023-03-22 15:54:41,836 INFO [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d5f4b1397e7e4f07a82dabc014dab6a9 columnFamilyName family1 2023-03-22 15:54:41,843 DEBUG [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/13f1a0d9d9ab43ceb16562e9f79afab9 2023-03-22 15:54:41,863 DEBUG [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/family1/27f0c2ace28a45b4a1ca97716314b03d_SeqId_53_ 2023-03-22 15:54:41,863 INFO [StoreOpener-d5f4b1397e7e4f07a82dabc014dab6a9-1] regionserver.HStore(310): Store=d5f4b1397e7e4f07a82dabc014dab6a9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:41,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:41,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:41,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:54:41,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d5f4b1397e7e4f07a82dabc014dab6a9; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60415693, jitterRate=-0.09973601996898651}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:54:41,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:54:41,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9., pid=53, masterSystemTime=1679500481829 2023-03-22 15:54:41,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:41,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:54:41,882 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=d5f4b1397e7e4f07a82dabc014dab6a9, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:41,882 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679500481882"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500481882"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500481882"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500481882"}]},"ts":"1679500481882"} 2023-03-22 15:54:41,885 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-22 15:54:41,885 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure d5f4b1397e7e4f07a82dabc014dab6a9, server=jenkins-hbase4.apache.org,38637,1679500416185 in 207 msec 2023-03-22 15:54:41,887 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-22 15:54:41,887 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=d5f4b1397e7e4f07a82dabc014dab6a9, REOPEN/MOVE in 534 msec 2023-03-22 15:54:41,890 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-22 15:54:41,890 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 539 msec 2023-03-22 15:54:41,892 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 980 msec 2023-03-22 15:54:41,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-22 15:54:41,894 INFO [Listener at localhost/43289] compactions.TestMobCompactor(278): alter status finished 2023-03-22 15:54:41,895 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-22 15:54:41,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-22 15:54:41,898 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-22 15:54:41,899 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501081899 2023-03-22 15:54:41,899 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:41,900 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:54:41,901 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-22 15:54:41,901 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:41,901 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x1485a1f1 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:41,939 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@306d8758, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:41,939 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:41,939 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:41,939 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:41,939 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:41,939 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:41,940 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:41,940 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:41,940 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-22 15:54:41,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-22 15:54:41,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:41,942 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-22 15:54:41,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 46 msec 2023-03-22 15:54:41,996 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=246 (was 268), OpenFileDescriptor=613 (was 644), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=374 (was 434), ProcessCount=170 (was 170), AvailableMemoryMB=6503 (was 6616) 2023-03-22 15:54:42,018 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=246, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=374, ProcessCount=170, AvailableMemoryMB=6502 2023-03-22 15:54:42,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-22 15:54:42,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:42,026 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:54:42,027 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:54:42,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-22 15:54:42,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-22 15:54:42,040 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:42,041 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e empty. 2023-03-22 15:54:42,042 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:42,042 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-22 15:54:42,063 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-22 15:54:42,065 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => dbbe80b35c71f0b8360fdcc3cdd8ef7e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:42,074 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:42,074 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing dbbe80b35c71f0b8360fdcc3cdd8ef7e, disabling compactions & flushes 2023-03-22 15:54:42,074 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:42,074 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:42,074 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. after waiting 0 ms 2023-03-22 15:54:42,074 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:42,077 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:42,077 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:42,079 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:54:42,080 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500482080"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500482080"}]},"ts":"1679500482080"} 2023-03-22 15:54:42,082 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-22 15:54:42,083 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:54:42,083 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500482083"}]},"ts":"1679500482083"} 2023-03-22 15:54:42,084 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-22 15:54:42,089 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, ASSIGN}] 2023-03-22 15:54:42,090 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, ASSIGN 2023-03-22 15:54:42,091 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:42,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-22 15:54:42,243 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:42,243 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500482243"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500482243"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500482243"}]},"ts":"1679500482243"} 2023-03-22 15:54:42,245 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:42,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-22 15:54:42,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:42,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dbbe80b35c71f0b8360fdcc3cdd8ef7e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:54:42,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:42,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:42,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:42,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:42,401 INFO [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:42,403 DEBUG [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:42,403 DEBUG [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:42,403 INFO [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dbbe80b35c71f0b8360fdcc3cdd8ef7e columnFamilyName family1 2023-03-22 15:54:42,405 INFO [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] regionserver.HStore(310): Store=dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:42,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:42,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:42,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:42,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:42,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dbbe80b35c71f0b8360fdcc3cdd8ef7e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73834722, jitterRate=0.10022309422492981}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:54:42,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:42,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e., pid=57, masterSystemTime=1679500482396 2023-03-22 15:54:42,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:42,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:42,415 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:42,415 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500482415"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500482415"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500482415"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500482415"}]},"ts":"1679500482415"} 2023-03-22 15:54:42,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-22 15:54:42,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185 in 173 msec 2023-03-22 15:54:42,420 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-22 15:54:42,420 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, ASSIGN in 329 msec 2023-03-22 15:54:42,421 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:54:42,421 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500482421"}]},"ts":"1679500482421"} 2023-03-22 15:54:42,422 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-22 15:54:42,424 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:54:42,425 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 403 msec 2023-03-22 15:54:42,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-22 15:54:42,633 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-22 15:54:42,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:42,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:42,671 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:42,671 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:42,671 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:42,671 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:42,672 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:42,672 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:42,673 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:42,673 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:42,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:42,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:42,674 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:42,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:42,674 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:42,674 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:42,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:42,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:42,675 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:42,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:42,684 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:42,684 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:42,684 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:42,692 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:42,692 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:42,692 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:42,692 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:42,693 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-22 15:54:43,154 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511302c789f002eb64d48be2bde1a03171d6f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511302c789f002eb64d48be2bde1a03171d6f 2023-03-22 15:54:43,561 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/a3f0a14bf68745cbb8da48ac0d25d063 2023-03-22 15:54:43,567 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/a3f0a14bf68745cbb8da48ac0d25d063 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a3f0a14bf68745cbb8da48ac0d25d063 2023-03-22 15:54:43,573 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a3f0a14bf68745cbb8da48ac0d25d063, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-22 15:54:43,575 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 882ms, sequenceid=5, compaction requested=false 2023-03-22 15:54:43,575 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:43,576 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:43,576 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:43,576 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:43,576 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:43,576 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,576 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:43,576 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:43,578 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,578 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:43,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:43,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:43,579 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,579 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:43,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:43,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:43,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:43,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35987112[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:43,581 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:43,581 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,582 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,582 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:43,582 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:43,582 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:43,583 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,583 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:43,584 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:43,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:43,584 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:43,584 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:43,584 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:43,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:43,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:43,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:43,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:43,587 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:43,587 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:43,587 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:43,587 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,587 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:43,590 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,590 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:43,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:43,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,590 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:43,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:43,591 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:43,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,593 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:43,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,600 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:43,600 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:43,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:43,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:43,600 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,600 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,600 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,600 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,600 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,600 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,600 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,601 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:43,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:43,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:43,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:43,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:43,601 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:43,601 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:43,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-22 15:54:43,701 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:43,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:43,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:43,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:43,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,708 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:43,708 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:43,708 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:43,708 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:43,709 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,709 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,710 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,710 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:43,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:43,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:43,710 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:43,710 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:43,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:43,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,711 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-22 15:54:43,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:43,712 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:43,712 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:43,712 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:43,712 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:43,712 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:43,712 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:43,712 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:43,712 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-22 15:54:44,136 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112843c083fa31be4ea39127320a127348d4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112843c083fa31be4ea39127320a127348d4 2023-03-22 15:54:44,544 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/dcfac32440f7499c98ea1156ac5572f9 2023-03-22 15:54:44,550 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/dcfac32440f7499c98ea1156ac5572f9 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/dcfac32440f7499c98ea1156ac5572f9 2023-03-22 15:54:44,554 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/dcfac32440f7499c98ea1156ac5572f9, entries=1, sequenceid=9, filesize=5.0 K 2023-03-22 15:54:44,555 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 843ms, sequenceid=9, compaction requested=false 2023-03-22 15:54:44,555 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:44,555 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:44,555 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:44,555 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:44,555 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,555 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:44,555 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:44,557 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:44,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:44,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a60092a[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:44,559 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:44,559 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,560 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,560 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:44,560 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:44,560 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:44,561 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,561 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:44,562 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:44,565 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:44,566 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:44,566 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:44,566 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,566 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:44,567 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,567 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:44,567 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,569 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:44,570 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,572 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:44,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:44,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:44,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:44,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:44,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:44,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,572 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,573 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:44,573 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:44,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:44,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:44,673 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:44,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:44,678 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:44,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:44,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,679 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:44,679 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:44,679 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:44,679 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:44,680 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,680 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,681 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:44,681 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:44,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:44,681 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,681 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:44,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:44,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,682 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,682 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-22 15:54:44,682 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,682 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:44,682 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:44,682 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:44,683 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:44,683 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:44,683 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:44,683 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:44,683 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:44,696 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512016efd9a4449fd443882f93c528e73cbaa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512016efd9a4449fd443882f93c528e73cbaa 2023-03-22 15:54:44,705 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/5a9e693b02b5427eb7a0de0acb5f102a 2023-03-22 15:54:44,710 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/5a9e693b02b5427eb7a0de0acb5f102a as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5a9e693b02b5427eb7a0de0acb5f102a 2023-03-22 15:54:44,714 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5a9e693b02b5427eb7a0de0acb5f102a, entries=1, sequenceid=13, filesize=5.0 K 2023-03-22 15:54:44,715 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 32ms, sequenceid=13, compaction requested=true 2023-03-22 15:54:44,715 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:44,715 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:44,715 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:44,715 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:44,715 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,716 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:44,716 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:44,718 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,718 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:44,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:44,718 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,719 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:44,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:44,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:44,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:44,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:44,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53e0dab0[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:44,720 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:44,720 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,721 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,721 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:44,721 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:44,721 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,721 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:44,721 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:44,722 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:44,723 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:44,722 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:44,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:44,723 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:44,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:44,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:44,725 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:44,725 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:44,725 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:44,725 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,725 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:44,727 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,727 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:44,727 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:44,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:44,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:44,728 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,730 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,734 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,734 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,734 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,734 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:44,734 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:44,735 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:44,735 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:44,735 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:44,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:44,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:44,735 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,735 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:44,735 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:44,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:44,735 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:44,836 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:44,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:44,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:44,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:44,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,848 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:44,848 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:44,848 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:44,848 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:44,848 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,848 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,849 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:44,849 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:44,850 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:44,850 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,850 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,851 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,851 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-22 15:54:44,851 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:44,851 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:44,851 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:44,851 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:45,266 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e5e3fc6c95094dad9edc62cf6ebde467 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205e5e3fc6c95094dad9edc62cf6ebde467 2023-03-22 15:54:45,272 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/f2e2654111bf4e1789f73f730dc0d472 2023-03-22 15:54:45,277 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/f2e2654111bf4e1789f73f730dc0d472 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/f2e2654111bf4e1789f73f730dc0d472 2023-03-22 15:54:45,281 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/f2e2654111bf4e1789f73f730dc0d472, entries=1, sequenceid=17, filesize=5.0 K 2023-03-22 15:54:45,282 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 431ms, sequenceid=17, compaction requested=true 2023-03-22 15:54:45,282 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:45,282 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:45,282 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:45,285 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,285 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:45,285 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,285 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:45,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:45,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c896d83[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:45,287 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:45,287 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,297 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,297 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:45,297 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:45,297 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:45,297 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,297 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:45,299 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,299 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:45,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:45,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:45,299 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:45,299 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:45,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:45,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:45,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:45,302 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:45,302 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:45,302 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:45,302 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,302 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:45,304 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,304 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:45,304 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:45,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:45,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:45,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:45,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:45,308 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:45,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,310 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:45,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,312 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,312 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,313 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,313 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,313 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,313 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,313 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,313 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:45,313 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-22 15:54:45,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:45,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:45,313 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:45,313 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:45,321 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:45,321 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:45,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:45,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:45,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:45,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:45,413 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:45,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:45,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:45,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:45,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,421 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:45,421 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:45,421 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:45,421 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:45,421 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,421 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,423 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:45,423 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:45,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:45,424 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,424 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:45,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:45,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-22 15:54:45,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,425 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:45,425 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:45,425 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:45,425 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:45,425 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:45,425 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:45,426 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:45,426 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:45,632 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:54:45,839 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512281e9b12af0c3a4c92a75f809c8daab7cc to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512281e9b12af0c3a4c92a75f809c8daab7cc 2023-03-22 15:54:45,845 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/7c4ded3ebcb948b6ab8132c5f75a76fd 2023-03-22 15:54:45,852 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/7c4ded3ebcb948b6ab8132c5f75a76fd as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/7c4ded3ebcb948b6ab8132c5f75a76fd 2023-03-22 15:54:45,856 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/7c4ded3ebcb948b6ab8132c5f75a76fd, entries=1, sequenceid=21, filesize=5.0 K 2023-03-22 15:54:45,857 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 431ms, sequenceid=21, compaction requested=true 2023-03-22 15:54:45,858 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:45,858 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:45,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:45,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:45,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:45,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:45,860 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,860 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:45,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:45,860 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,860 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:45,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:45,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:45,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:45,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:45,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7aef8de1[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:45,862 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:45,862 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,863 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:45,864 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,864 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:45,865 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:45,865 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,866 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:45,866 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:45,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:45,869 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:45,869 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:45,869 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:45,869 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,869 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:45,872 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,872 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:45,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,873 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:45,874 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:45,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,876 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:45,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,879 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:45,879 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:45,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:45,880 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,880 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,880 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,880 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,880 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,880 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,880 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:45,880 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:45,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:45,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:45,881 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:45,881 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:45,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-22 15:54:45,981 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:45,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:45,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:45,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:45,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,989 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:45,989 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:45,989 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:45,989 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:45,989 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,989 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,991 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:45,991 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:45,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:45,991 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:45,991 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:45,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:45,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-22 15:54:45,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:45,993 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:45,993 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:45,993 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:45,993 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:45,993 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:45,993 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:45,993 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:45,993 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:46,409 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231b7b70d866bd146d1be7b5632719f138b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231b7b70d866bd146d1be7b5632719f138b 2023-03-22 15:54:46,415 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/2cd377f7723b4641bb937ac7444d6da7 2023-03-22 15:54:46,420 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/2cd377f7723b4641bb937ac7444d6da7 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/2cd377f7723b4641bb937ac7444d6da7 2023-03-22 15:54:46,425 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/2cd377f7723b4641bb937ac7444d6da7, entries=1, sequenceid=25, filesize=5.0 K 2023-03-22 15:54:46,426 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 433ms, sequenceid=25, compaction requested=true 2023-03-22 15:54:46,426 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:46,427 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:46,427 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:46,427 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:46,427 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,427 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:46,427 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:46,428 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,428 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:46,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:46,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60dce2cf[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:46,430 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:46,430 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,433 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,433 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:46,433 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:46,433 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:46,433 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,433 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:46,435 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:46,435 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:46,436 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:46,436 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:46,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:46,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:46,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:46,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:46,441 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:46,441 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:46,441 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:46,441 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,441 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:46,443 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,443 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:46,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:46,443 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:46,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:46,444 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:46,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,445 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:46,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,451 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:46,451 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:46,451 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:46,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:46,451 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-22 15:54:46,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:46,451 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:46,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:46,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:46,552 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:46,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:46,557 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:46,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:46,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,584 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:46,584 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:46,584 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:46,584 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:46,585 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,585 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,590 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:46,590 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:46,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:46,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,591 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:46,591 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:46,591 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:46,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,592 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-22 15:54:46,592 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:46,593 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:46,593 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:46,593 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:46,593 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:46,593 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:46,593 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:46,593 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:46,593 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:47,011 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601017d4c90f5a1c24899a53917154edcdd22 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601017d4c90f5a1c24899a53917154edcdd22 2023-03-22 15:54:47,418 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/0bbb4c2cde3a451b9c15ca93cb42df71 2023-03-22 15:54:47,423 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/0bbb4c2cde3a451b9c15ca93cb42df71 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/0bbb4c2cde3a451b9c15ca93cb42df71 2023-03-22 15:54:47,428 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/0bbb4c2cde3a451b9c15ca93cb42df71, entries=1, sequenceid=29, filesize=5.0 K 2023-03-22 15:54:47,429 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 836ms, sequenceid=29, compaction requested=true 2023-03-22 15:54:47,429 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:47,429 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:47,429 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:47,429 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:47,429 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,429 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:47,429 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:47,431 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,431 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:47,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:47,431 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,431 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:47,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:47,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:47,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:47,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:47,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@353c913e[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:47,435 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:47,436 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,437 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,437 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:47,437 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:47,438 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:47,438 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,438 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:47,440 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:47,440 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:47,440 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,440 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:47,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:47,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:47,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:47,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:47,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:47,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:47,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:47,444 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:47,444 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:47,444 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:47,444 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,444 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:47,446 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,446 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,446 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:47,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:47,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,447 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:47,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:47,449 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:47,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:47,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:47,451 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:47,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:47,451 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:47,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,451 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:47,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:47,451 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,452 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,452 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,452 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,452 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,452 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,452 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:47,452 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:47,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-22 15:54:47,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:47,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:47,552 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:47,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:47,557 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:47,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:47,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,559 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:47,559 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:47,559 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:47,559 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:47,560 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,560 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,561 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:47,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:47,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:47,561 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,562 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:47,562 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:47,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:47,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-22 15:54:47,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:47,564 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:47,564 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:47,564 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:47,564 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:47,564 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:47,564 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:47,564 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:47,564 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:47,983 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103eec6f563c37143279083d24183b30565 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103eec6f563c37143279083d24183b30565 2023-03-22 15:54:48,389 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/cc338db33b52443fb625cb26b1eb2810 2023-03-22 15:54:48,405 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/cc338db33b52443fb625cb26b1eb2810 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/cc338db33b52443fb625cb26b1eb2810 2023-03-22 15:54:48,411 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/cc338db33b52443fb625cb26b1eb2810, entries=1, sequenceid=33, filesize=5.0 K 2023-03-22 15:54:48,412 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 848ms, sequenceid=33, compaction requested=true 2023-03-22 15:54:48,413 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:48,413 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:48,413 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:48,413 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:48,413 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,413 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:48,413 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:48,415 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,415 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:48,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:48,416 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,416 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:48,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:48,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:48,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:48,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:48,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@721e076f[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:48,418 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:48,418 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,419 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,420 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:48,420 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:48,420 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:48,420 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,420 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:48,421 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:48,421 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,421 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:48,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:48,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:48,421 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:48,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:48,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:48,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:48,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:48,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:48,424 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:48,424 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:48,424 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:48,424 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,424 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:48,428 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,428 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:48,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,428 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:48,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:48,429 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:48,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,431 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:48,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,434 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,434 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,434 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:48,434 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:48,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-22 15:54:48,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:48,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:48,434 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,434 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:48,435 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,435 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,434 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:48,435 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:48,435 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:48,435 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:48,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:48,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:48,535 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:48,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:48,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:48,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:48,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,542 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:48,542 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:48,542 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:48,542 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:48,543 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,543 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,544 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:48,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:48,544 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:48,544 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,545 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:48,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,545 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:48,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,546 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-22 15:54:48,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:48,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:48,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:48,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:48,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:48,546 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:48,546 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:48,546 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:48,570 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509076337f414746141c88e6dc31a9eaae91f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509076337f414746141c88e6dc31a9eaae91f 2023-03-22 15:54:48,600 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/3b0fedd4a3b74421911ca6304a2f7b73 2023-03-22 15:54:48,606 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/3b0fedd4a3b74421911ca6304a2f7b73 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/3b0fedd4a3b74421911ca6304a2f7b73 2023-03-22 15:54:48,613 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/3b0fedd4a3b74421911ca6304a2f7b73, entries=1, sequenceid=37, filesize=5.0 K 2023-03-22 15:54:48,613 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 67ms, sequenceid=37, compaction requested=true 2023-03-22 15:54:48,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:48,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:48,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:48,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:48,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:48,614 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:48,616 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,616 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:48,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:48,617 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,617 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:48,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:48,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:48,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:48,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:48,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74181c48[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:48,619 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:48,619 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,621 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,621 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:48,621 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:48,621 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:48,621 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,622 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:48,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:48,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:48,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:48,623 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:48,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:48,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:48,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:48,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:48,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:48,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:48,626 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:48,626 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:48,626 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:48,626 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,626 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:48,628 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,628 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:48,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,628 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:48,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:48,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:48,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:48,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,629 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:48,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:48,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,630 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:48,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,632 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,632 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,632 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,632 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:48,633 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:48,632 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:48,633 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:48,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:48,633 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-22 15:54:48,633 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:48,633 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:48,633 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,633 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,633 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:48,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:48,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:48,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:48,733 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:48,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:48,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:48,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-22 15:54:48,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,750 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:48,750 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:48,751 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-22 15:54:48,751 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-22 15:54:48,751 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,751 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,752 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:48,752 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:48,753 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:48,753 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:48,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,754 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,754 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-22 15:54:48,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-22 15:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-22 15:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:48,754 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:48,755 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. started... 2023-03-22 15:54:48,755 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dbbe80b35c71f0b8360fdcc3cdd8ef7e 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:54:49,171 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120730ec904be204cf0bb7ba976c77ebc03 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120730ec904be204cf0bb7ba976c77ebc03 2023-03-22 15:54:49,581 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/d20b05577cf44829bcb7c3d41a688000 2023-03-22 15:54:49,586 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/d20b05577cf44829bcb7c3d41a688000 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/d20b05577cf44829bcb7c3d41a688000 2023-03-22 15:54:49,590 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/d20b05577cf44829bcb7c3d41a688000, entries=1, sequenceid=41, filesize=5.0 K 2023-03-22 15:54:49,591 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for dbbe80b35c71f0b8360fdcc3cdd8ef7e in 836ms, sequenceid=41, compaction requested=true 2023-03-22 15:54:49,591 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:49,591 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:49,591 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:54:49,591 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:54:49,591 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,591 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-22 15:54:49,591 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-22 15:54:49,593 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,593 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:49,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:49,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:49,594 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,594 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:54:49,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:49,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:49,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-22 15:54:49,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73d5429e[Count = 0] remaining members to acquire global barrier 2023-03-22 15:54:49,595 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-22 15:54:49,595 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,597 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,597 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-22 15:54:49,597 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-22 15:54:49,597 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:54:49,597 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,597 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:54:49,599 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,600 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-22 15:54:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:49,600 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:49,600 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-22 15:54:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:49,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:49,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:54:49,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-22 15:54:49,602 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-22 15:54:49,602 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:54:49,602 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:54:49,602 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,602 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:54:49,604 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:54:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:54:49,604 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,604 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:49,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:54:49,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:49,604 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:54:49,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,606 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:54:49,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,608 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,608 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,608 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:49,608 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:54:49,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:49,608 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:54:49,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-22 15:54:49,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:54:49,608 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:49,609 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:49,609 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,609 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,609 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,609 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-22 15:54:49,609 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:54:49,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:54:49,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:54:49,709 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:54:49,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:54:49,711 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:49,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-22 15:54:49,714 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-22 15:54:49,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:49,714 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501089714 2023-03-22 15:54:49,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:54:49,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-22 15:54:49,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:49,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x317f0b25 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:49,724 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@75717156, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:49,724 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:50,620 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103aa603d861bac45dfa6499c067847d969 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103aa603d861bac45dfa6499c067847d969 2023-03-22 15:54:50,621 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:50,635 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:54:50,636 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33268, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:54:50,647 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:54:50,650 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59538, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:54:50,677 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a24d2ad20c55441285fead226bec9d16 first=Optional[r3] last=Optional[r6] 2023-03-22 15:54:50,678 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a24d2ad20c55441285fead226bec9d16}] 2023-03-22 15:54:50,684 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a24d2ad20c55441285fead226bec9d16 for inclusion in dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:50,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-22 15:54:50,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:50,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:50,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a24d2ad20c55441285fead226bec9d16 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r5clfbkoepdi3k6ce5lqcvp53ac7utr5icum5re2ji68m8rm15n801slenv57cnc/family1/a24d2ad20c55441285fead226bec9d16 2023-03-22 15:54:50,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r5clfbkoepdi3k6ce5lqcvp53ac7utr5icum5re2ji68m8rm15n801slenv57cnc/family1/a24d2ad20c55441285fead226bec9d16 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5b591118f9bd4b0189af96ea8a3b58cd_SeqId_44_ 2023-03-22 15:54:50,695 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r5clfbkoepdi3k6ce5lqcvp53ac7utr5icum5re2ji68m8rm15n801slenv57cnc/family1/a24d2ad20c55441285fead226bec9d16 into dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5b591118f9bd4b0189af96ea8a3b58cd_SeqId_44_ - updating store file list. 2023-03-22 15:54:50,699 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5b591118f9bd4b0189af96ea8a3b58cd_SeqId_44_ into dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:50,699 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r5clfbkoepdi3k6ce5lqcvp53ac7utr5icum5re2ji68m8rm15n801slenv57cnc/family1/a24d2ad20c55441285fead226bec9d16 into dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5b591118f9bd4b0189af96ea8a3b58cd_SeqId_44_) 2023-03-22 15:54:50,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r5clfbkoepdi3k6ce5lqcvp53ac7utr5icum5re2ji68m8rm15n801slenv57cnc/family1/a24d2ad20c55441285fead226bec9d16 2023-03-22 15:54:50,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-22 15:54:50,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-22 15:54:50,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 is initiating minor compaction 2023-03-22 15:54:50,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7693be30; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:50,704 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:50,704 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/dcfac32440f7499c98ea1156ac5572f9, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5a9e693b02b5427eb7a0de0acb5f102a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/f2e2654111bf4e1789f73f730dc0d472, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/7c4ded3ebcb948b6ab8132c5f75a76fd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/2cd377f7723b4641bb937ac7444d6da7, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/0bbb4c2cde3a451b9c15ca93cb42df71, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/cc338db33b52443fb625cb26b1eb2810, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/3b0fedd4a3b74421911ca6304a2f7b73, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/d20b05577cf44829bcb7c3d41a688000, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5b591118f9bd4b0189af96ea8a3b58cd_SeqId_44_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp, totalSize=50.3 K 2023-03-22 15:54:50,704 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting dcfac32440f7499c98ea1156ac5572f9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-22 15:54:50,704 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 5a9e693b02b5427eb7a0de0acb5f102a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-22 15:54:50,704 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting f2e2654111bf4e1789f73f730dc0d472, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-22 15:54:50,704 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 7c4ded3ebcb948b6ab8132c5f75a76fd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-22 15:54:50,704 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 2cd377f7723b4641bb937ac7444d6da7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-22 15:54:50,704 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 0bbb4c2cde3a451b9c15ca93cb42df71, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-22 15:54:50,704 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting cc338db33b52443fb625cb26b1eb2810, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-22 15:54:50,705 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 3b0fedd4a3b74421911ca6304a2f7b73, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-22 15:54:50,705 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting d20b05577cf44829bcb7c3d41a688000, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-22 15:54:50,705 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 5b591118f9bd4b0189af96ea8a3b58cd_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-22 15:54:50,709 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:50,710 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:50,712 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512281e9b12af0c3a4c92a75f809c8daab7cc to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512281e9b12af0c3a4c92a75f809c8daab7cc 2023-03-22 15:54:50,713 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231b7b70d866bd146d1be7b5632719f138b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231b7b70d866bd146d1be7b5632719f138b 2023-03-22 15:54:50,715 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601017d4c90f5a1c24899a53917154edcdd22 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601017d4c90f5a1c24899a53917154edcdd22 2023-03-22 15:54:50,716 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103eec6f563c37143279083d24183b30565 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103eec6f563c37143279083d24183b30565 2023-03-22 15:54:50,716 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-22 15:54:50,716 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-22 15:54:50,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:50,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:50,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:50,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:50,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:50,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:50,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:50,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:50,746 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): dbbe80b35c71f0b8360fdcc3cdd8ef7e#family1#compaction#72 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:54:51,613 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c2e5116752c541d68c6166d8aa2649fc to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205c2e5116752c541d68c6166d8aa2649fc 2023-03-22 15:54:51,614 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/10e0a138a1f4442e8dc118c9f1eb32fa as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/10e0a138a1f4442e8dc118c9f1eb32fa 2023-03-22 15:54:51,614 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:51,634 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 of dbbe80b35c71f0b8360fdcc3cdd8ef7e into 10e0a138a1f4442e8dc118c9f1eb32fa(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:51,634 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:51,634 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1679500490704; duration=0sec 2023-03-22 15:54:51,634 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:51,643 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/987fdbad80a84d328123e407072c9b52 first=Optional[r00] last=Optional[r2] 2023-03-22 15:54:51,644 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/987fdbad80a84d328123e407072c9b52}] 2023-03-22 15:54:51,646 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/987fdbad80a84d328123e407072c9b52 for inclusion in dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:51,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-22 15:54:51,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:51,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:51,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/987fdbad80a84d328123e407072c9b52 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vmdegl2unacg6rsac67p3966al3gdhhe77f7339l32vi7ufslvb50t9igu6n1mje/family1/987fdbad80a84d328123e407072c9b52 2023-03-22 15:54:51,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vmdegl2unacg6rsac67p3966al3gdhhe77f7339l32vi7ufslvb50t9igu6n1mje/family1/987fdbad80a84d328123e407072c9b52 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/4800f018c1894f7aa0441ab18eee32bd_SeqId_47_ 2023-03-22 15:54:51,656 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vmdegl2unacg6rsac67p3966al3gdhhe77f7339l32vi7ufslvb50t9igu6n1mje/family1/987fdbad80a84d328123e407072c9b52 into dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/4800f018c1894f7aa0441ab18eee32bd_SeqId_47_ - updating store file list. 2023-03-22 15:54:51,660 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/4800f018c1894f7aa0441ab18eee32bd_SeqId_47_ into dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:51,660 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vmdegl2unacg6rsac67p3966al3gdhhe77f7339l32vi7ufslvb50t9igu6n1mje/family1/987fdbad80a84d328123e407072c9b52 into dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/4800f018c1894f7aa0441ab18eee32bd_SeqId_47_) 2023-03-22 15:54:51,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vmdegl2unacg6rsac67p3966al3gdhhe77f7339l32vi7ufslvb50t9igu6n1mje/family1/987fdbad80a84d328123e407072c9b52 2023-03-22 15:54:51,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-22 15:54:51,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-22 15:54:51,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 is initiating minor compaction (all files) 2023-03-22 15:54:51,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2f5e96df; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:51,662 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:51,662 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a3f0a14bf68745cbb8da48ac0d25d063, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/10e0a138a1f4442e8dc118c9f1eb32fa, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/4800f018c1894f7aa0441ab18eee32bd_SeqId_47_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp, totalSize=388.1 K 2023-03-22 15:54:51,664 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting a3f0a14bf68745cbb8da48ac0d25d063, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840482 2023-03-22 15:54:51,664 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 10e0a138a1f4442e8dc118c9f1eb32fa, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240482 2023-03-22 15:54:51,665 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 4800f018c1894f7aa0441ab18eee32bd_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840482 2023-03-22 15:54:51,667 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:51,668 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:51,671 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511302c789f002eb64d48be2bde1a03171d6f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511302c789f002eb64d48be2bde1a03171d6f 2023-03-22 15:54:51,672 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512016efd9a4449fd443882f93c528e73cbaa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512016efd9a4449fd443882f93c528e73cbaa 2023-03-22 15:54:51,674 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205e5e3fc6c95094dad9edc62cf6ebde467 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205e5e3fc6c95094dad9edc62cf6ebde467 2023-03-22 15:54:51,674 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-22 15:54:51,674 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-22 15:54:51,674 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:51,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:51,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:51,676 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-22 15:54:51,677 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.9640 sec 2023-03-22 15:54:51,682 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): dbbe80b35c71f0b8360fdcc3cdd8ef7e#family1#compaction#73 average throughput is 19.96 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:54:51,691 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/dcfac32440f7499c98ea1156ac5572f9, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5a9e693b02b5427eb7a0de0acb5f102a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/f2e2654111bf4e1789f73f730dc0d472, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/7c4ded3ebcb948b6ab8132c5f75a76fd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/2cd377f7723b4641bb937ac7444d6da7, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/0bbb4c2cde3a451b9c15ca93cb42df71, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/cc338db33b52443fb625cb26b1eb2810, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/3b0fedd4a3b74421911ca6304a2f7b73, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/d20b05577cf44829bcb7c3d41a688000, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5b591118f9bd4b0189af96ea8a3b58cd_SeqId_44_] to archive 2023-03-22 15:54:51,692 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:51,695 WARN [IPC Server handler 4 on default port 33875] namenode.FSDirRenameOp(189): DIR* FSDirectory.unprotectedRenameTo: failed to rename /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/dcfac32440f7499c98ea1156ac5572f9 to /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/dcfac32440f7499c98ea1156ac5572f9 because destination's parent does not exist 2023-03-22 15:54:51,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(557): Created archive directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:51,698 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/dcfac32440f7499c98ea1156ac5572f9 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/dcfac32440f7499c98ea1156ac5572f9 2023-03-22 15:54:51,709 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5a9e693b02b5427eb7a0de0acb5f102a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5a9e693b02b5427eb7a0de0acb5f102a 2023-03-22 15:54:51,712 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/f2e2654111bf4e1789f73f730dc0d472 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/f2e2654111bf4e1789f73f730dc0d472 2023-03-22 15:54:51,720 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-03-22 15:54:51,722 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/7c4ded3ebcb948b6ab8132c5f75a76fd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/7c4ded3ebcb948b6ab8132c5f75a76fd 2023-03-22 15:54:51,725 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/2cd377f7723b4641bb937ac7444d6da7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/2cd377f7723b4641bb937ac7444d6da7 2023-03-22 15:54:51,730 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/0bbb4c2cde3a451b9c15ca93cb42df71 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/0bbb4c2cde3a451b9c15ca93cb42df71 2023-03-22 15:54:51,732 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/cc338db33b52443fb625cb26b1eb2810 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/cc338db33b52443fb625cb26b1eb2810 2023-03-22 15:54:51,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/3b0fedd4a3b74421911ca6304a2f7b73 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/3b0fedd4a3b74421911ca6304a2f7b73 2023-03-22 15:54:51,735 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/d20b05577cf44829bcb7c3d41a688000 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/d20b05577cf44829bcb7c3d41a688000 2023-03-22 15:54:51,737 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5b591118f9bd4b0189af96ea8a3b58cd_SeqId_44_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/5b591118f9bd4b0189af96ea8a3b58cd_SeqId_44_ 2023-03-22 15:54:51,754 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/.tmp/family1/55e68c7d058241438beb19c32e0b8257 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/55e68c7d058241438beb19c32e0b8257 2023-03-22 15:54:51,774 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 of dbbe80b35c71f0b8360fdcc3cdd8ef7e into 55e68c7d058241438beb19c32e0b8257(size=192.4 K), total size for store is 192.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:51,774 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:51,774 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1679500491662; duration=0sec 2023-03-22 15:54:51,774 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:51,800 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a3f0a14bf68745cbb8da48ac0d25d063, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/10e0a138a1f4442e8dc118c9f1eb32fa, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/4800f018c1894f7aa0441ab18eee32bd_SeqId_47_] to archive 2023-03-22 15:54:51,801 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:51,804 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a3f0a14bf68745cbb8da48ac0d25d063 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a3f0a14bf68745cbb8da48ac0d25d063 2023-03-22 15:54:51,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/10e0a138a1f4442e8dc118c9f1eb32fa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/10e0a138a1f4442e8dc118c9f1eb32fa 2023-03-22 15:54:51,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-22 15:54:51,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:51,814 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/4800f018c1894f7aa0441ab18eee32bd_SeqId_47_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/4800f018c1894f7aa0441ab18eee32bd_SeqId_47_ 2023-03-22 15:54:52,244 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-22 15:54:52,246 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-22 15:54:52,246 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-22 15:54:52,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-22 15:54:52,257 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, REOPEN/MOVE}] 2023-03-22 15:54:52,257 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, REOPEN/MOVE 2023-03-22 15:54:52,258 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:52,258 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500492258"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500492258"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500492258"}]},"ts":"1679500492258"} 2023-03-22 15:54:52,260 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:52,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:52,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dbbe80b35c71f0b8360fdcc3cdd8ef7e, disabling compactions & flushes 2023-03-22 15:54:52,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:52,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:52,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. after waiting 0 ms 2023-03-22 15:54:52,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:52,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-22 15:54:52,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:52,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:52,419 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: dbbe80b35c71f0b8360fdcc3cdd8ef7e to self. 2023-03-22 15:54:52,421 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=CLOSED 2023-03-22 15:54:52,422 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500492421"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500492421"}]},"ts":"1679500492421"} 2023-03-22 15:54:52,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:52,425 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-22 15:54:52,425 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185 in 163 msec 2023-03-22 15:54:52,425 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=true 2023-03-22 15:54:52,576 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:52,576 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500492576"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500492576"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500492576"}]},"ts":"1679500492576"} 2023-03-22 15:54:52,578 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:52,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:52,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dbbe80b35c71f0b8360fdcc3cdd8ef7e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:54:52,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:52,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:52,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:52,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:52,735 INFO [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:52,736 DEBUG [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:52,736 DEBUG [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:52,736 INFO [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dbbe80b35c71f0b8360fdcc3cdd8ef7e columnFamilyName family1 2023-03-22 15:54:52,741 DEBUG [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/55e68c7d058241438beb19c32e0b8257 2023-03-22 15:54:52,742 INFO [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] regionserver.HStore(310): Store=dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:52,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:52,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:52,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:52,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dbbe80b35c71f0b8360fdcc3cdd8ef7e; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59941123, jitterRate=-0.1068076640367508}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:54:52,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:52,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e., pid=63, masterSystemTime=1679500492730 2023-03-22 15:54:52,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:52,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:52,750 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:52,750 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500492750"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500492750"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500492750"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500492750"}]},"ts":"1679500492750"} 2023-03-22 15:54:52,753 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-22 15:54:52,753 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185 in 173 msec 2023-03-22 15:54:52,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-22 15:54:52,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, REOPEN/MOVE in 497 msec 2023-03-22 15:54:52,756 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-22 15:54:52,756 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 507 msec 2023-03-22 15:54:52,758 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 945 msec 2023-03-22 15:54:52,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-22 15:54:52,761 INFO [Listener at localhost/43289] compactions.TestMobCompactor(278): alter status finished 2023-03-22 15:54:52,763 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:52,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-22 15:54:52,769 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-22 15:54:52,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:52,770 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501092770 2023-03-22 15:54:52,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:54:52,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-22 15:54:52,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:52,772 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x18c333b6 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:52,787 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70bc2b83, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:52,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:52,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:52,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:52,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:52,788 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,219 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f891a001e5ac446bb28265f0cb6d9051 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f891a001e5ac446bb28265f0cb6d9051 2023-03-22 15:54:53,220 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:53,238 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:54:53,240 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33276, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:54:53,252 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:54:53,254 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59542, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:54:53,285 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/078d4fae2f524ed29ad897ef37d7e811 first=Optional[r05] last=Optional[r8] 2023-03-22 15:54:53,286 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/078d4fae2f524ed29ad897ef37d7e811}] 2023-03-22 15:54:53,291 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/078d4fae2f524ed29ad897ef37d7e811 for inclusion in dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:53,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-22 15:54:53,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:53,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:53,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/078d4fae2f524ed29ad897ef37d7e811 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emo4m8gqs46i5nb4jo2h5r3b10n6e5r9nnvjat0d05bf90n6b30pmgoqgubgv1b5/family1/078d4fae2f524ed29ad897ef37d7e811 2023-03-22 15:54:53,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emo4m8gqs46i5nb4jo2h5r3b10n6e5r9nnvjat0d05bf90n6b30pmgoqgubgv1b5/family1/078d4fae2f524ed29ad897ef37d7e811 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a0c93193541641249a799e72dde3361e_SeqId_53_ 2023-03-22 15:54:53,299 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emo4m8gqs46i5nb4jo2h5r3b10n6e5r9nnvjat0d05bf90n6b30pmgoqgubgv1b5/family1/078d4fae2f524ed29ad897ef37d7e811 into dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a0c93193541641249a799e72dde3361e_SeqId_53_ - updating store file list. 2023-03-22 15:54:53,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a0c93193541641249a799e72dde3361e_SeqId_53_ into dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:53,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emo4m8gqs46i5nb4jo2h5r3b10n6e5r9nnvjat0d05bf90n6b30pmgoqgubgv1b5/family1/078d4fae2f524ed29ad897ef37d7e811 into dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a0c93193541641249a799e72dde3361e_SeqId_53_) 2023-03-22 15:54:53,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__emo4m8gqs46i5nb4jo2h5r3b10n6e5r9nnvjat0d05bf90n6b30pmgoqgubgv1b5/family1/078d4fae2f524ed29ad897ef37d7e811 2023-03-22 15:54:53,308 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:53,308 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:53,310 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120730ec904be204cf0bb7ba976c77ebc03 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120730ec904be204cf0bb7ba976c77ebc03 2023-03-22 15:54:53,311 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112843c083fa31be4ea39127320a127348d4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112843c083fa31be4ea39127320a127348d4 2023-03-22 15:54:53,311 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-22 15:54:53,311 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-22 15:54:53,312 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,312 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,312 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:53,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:53,313 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-22 15:54:53,315 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 549 msec 2023-03-22 15:54:53,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-22 15:54:53,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:53,456 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-22 15:54:53,457 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-22 15:54:53,457 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-22 15:54:53,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-22 15:54:53,463 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, REOPEN/MOVE}] 2023-03-22 15:54:53,464 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, REOPEN/MOVE 2023-03-22 15:54:53,464 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:53,464 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500493464"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500493464"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500493464"}]},"ts":"1679500493464"} 2023-03-22 15:54:53,466 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:53,618 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:53,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dbbe80b35c71f0b8360fdcc3cdd8ef7e, disabling compactions & flushes 2023-03-22 15:54:53,618 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:53,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:53,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. after waiting 0 ms 2023-03-22 15:54:53,618 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:53,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-22 15:54:53,624 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:53,624 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:53,624 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: dbbe80b35c71f0b8360fdcc3cdd8ef7e to self. 2023-03-22 15:54:53,626 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:53,626 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=CLOSED 2023-03-22 15:54:53,626 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500493626"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500493626"}]},"ts":"1679500493626"} 2023-03-22 15:54:53,630 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-22 15:54:53,630 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185 in 161 msec 2023-03-22 15:54:53,630 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=true 2023-03-22 15:54:53,781 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:53,781 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500493781"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500493781"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500493781"}]},"ts":"1679500493781"} 2023-03-22 15:54:53,783 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:53,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:53,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dbbe80b35c71f0b8360fdcc3cdd8ef7e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:54:53,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:53,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:53,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:53,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:53,939 INFO [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:53,940 DEBUG [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:53,941 DEBUG [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1 2023-03-22 15:54:53,941 INFO [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dbbe80b35c71f0b8360fdcc3cdd8ef7e columnFamilyName family1 2023-03-22 15:54:53,946 DEBUG [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/55e68c7d058241438beb19c32e0b8257 2023-03-22 15:54:53,949 DEBUG [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1/a0c93193541641249a799e72dde3361e_SeqId_53_ 2023-03-22 15:54:53,950 INFO [StoreOpener-dbbe80b35c71f0b8360fdcc3cdd8ef7e-1] regionserver.HStore(310): Store=dbbe80b35c71f0b8360fdcc3cdd8ef7e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:53,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:53,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:53,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:54:53,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dbbe80b35c71f0b8360fdcc3cdd8ef7e; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62761792, jitterRate=-0.06477642059326172}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:54:53,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:54:53,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e., pid=69, masterSystemTime=1679500493935 2023-03-22 15:54:53,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:53,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:54:53,957 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=dbbe80b35c71f0b8360fdcc3cdd8ef7e, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:53,957 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679500493957"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500493957"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500493957"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500493957"}]},"ts":"1679500493957"} 2023-03-22 15:54:53,960 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-22 15:54:53,960 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure dbbe80b35c71f0b8360fdcc3cdd8ef7e, server=jenkins-hbase4.apache.org,38637,1679500416185 in 176 msec 2023-03-22 15:54:53,962 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-22 15:54:53,962 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=dbbe80b35c71f0b8360fdcc3cdd8ef7e, REOPEN/MOVE in 497 msec 2023-03-22 15:54:53,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-22 15:54:53,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 502 msec 2023-03-22 15:54:53,967 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 570 msec 2023-03-22 15:54:53,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-22 15:54:53,969 INFO [Listener at localhost/43289] compactions.TestMobCompactor(278): alter status finished 2023-03-22 15:54:53,970 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-22 15:54:53,972 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-22 15:54:53,973 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-22 15:54:53,973 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:53,973 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501093973 2023-03-22 15:54:53,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:54:53,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-22 15:54:53,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:53,975 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x61c0fd3a to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:53,982 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79a267d6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:53,982 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,982 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,982 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,982 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,982 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,982 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,982 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,983 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-22 15:54:53,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-22 15:54:53,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:53,984 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-22 15:54:53,986 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 13 msec 2023-03-22 15:54:54,054 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=252 (was 246) - Thread LEAK? -, OpenFileDescriptor=635 (was 613) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=394 (was 374) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6479 (was 6502) 2023-03-22 15:54:54,072 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=252, OpenFileDescriptor=635, MaxFileDescriptor=60000, SystemLoadAverage=394, ProcessCount=170, AvailableMemoryMB=6478 2023-03-22 15:54:54,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-22 15:54:54,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-22 15:54:54,078 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:54:54,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-22 15:54:54,079 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:54:54,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-22 15:54:54,081 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da 2023-03-22 15:54:54,081 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da empty. 2023-03-22 15:54:54,082 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da 2023-03-22 15:54:54,082 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-22 15:54:54,094 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-22 15:54:54,096 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => f25498a0985673b023870241e50965da, NAME => 'testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:54,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-22 15:54:54,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-22 15:54:54,516 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:54,516 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing f25498a0985673b023870241e50965da, disabling compactions & flushes 2023-03-22 15:54:54,516 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:54,516 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:54,516 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. after waiting 0 ms 2023-03-22 15:54:54,516 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:54,516 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:54,516 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for f25498a0985673b023870241e50965da: 2023-03-22 15:54:54,518 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:54:54,519 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679500494519"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500494519"}]},"ts":"1679500494519"} 2023-03-22 15:54:54,521 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-22 15:54:54,521 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:54:54,521 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500494521"}]},"ts":"1679500494521"} 2023-03-22 15:54:54,522 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-22 15:54:54,526 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f25498a0985673b023870241e50965da, ASSIGN}] 2023-03-22 15:54:54,527 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f25498a0985673b023870241e50965da, ASSIGN 2023-03-22 15:54:54,528 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=f25498a0985673b023870241e50965da, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:54,679 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=f25498a0985673b023870241e50965da, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:54,679 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679500494679"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500494679"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500494679"}]},"ts":"1679500494679"} 2023-03-22 15:54:54,681 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure f25498a0985673b023870241e50965da, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:54,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-22 15:54:54,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:54,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f25498a0985673b023870241e50965da, NAME => 'testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:54:54,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-22 15:54:54,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-22 15:54:54,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction f25498a0985673b023870241e50965da 2023-03-22 15:54:54,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:54,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f25498a0985673b023870241e50965da 2023-03-22 15:54:54,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f25498a0985673b023870241e50965da 2023-03-22 15:54:54,839 INFO [StoreOpener-f25498a0985673b023870241e50965da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region f25498a0985673b023870241e50965da 2023-03-22 15:54:54,840 DEBUG [StoreOpener-f25498a0985673b023870241e50965da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1 2023-03-22 15:54:54,840 DEBUG [StoreOpener-f25498a0985673b023870241e50965da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1 2023-03-22 15:54:54,841 INFO [StoreOpener-f25498a0985673b023870241e50965da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f25498a0985673b023870241e50965da columnFamilyName f1 2023-03-22 15:54:54,842 INFO [StoreOpener-f25498a0985673b023870241e50965da-1] regionserver.HStore(310): Store=f25498a0985673b023870241e50965da/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:54,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da 2023-03-22 15:54:54,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da 2023-03-22 15:54:54,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f25498a0985673b023870241e50965da 2023-03-22 15:54:54,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:54,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f25498a0985673b023870241e50965da; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67694773, jitterRate=0.008730724453926086}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:54:54,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f25498a0985673b023870241e50965da: 2023-03-22 15:54:54,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da., pid=73, masterSystemTime=1679500494832 2023-03-22 15:54:54,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:54,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:54,852 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=f25498a0985673b023870241e50965da, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:54,852 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679500494852"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500494852"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500494852"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500494852"}]},"ts":"1679500494852"} 2023-03-22 15:54:54,855 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-22 15:54:54,855 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure f25498a0985673b023870241e50965da, server=jenkins-hbase4.apache.org,38637,1679500416185 in 173 msec 2023-03-22 15:54:54,857 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-22 15:54:54,857 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=f25498a0985673b023870241e50965da, ASSIGN in 329 msec 2023-03-22 15:54:54,857 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:54:54,857 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500494857"}]},"ts":"1679500494857"} 2023-03-22 15:54:54,858 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-22 15:54:54,861 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:54:54,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 787 msec 2023-03-22 15:54:55,105 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-22 15:54:55,106 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro Metrics about HBase RegionObservers 2023-03-22 15:54:55,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-22 15:54:55,184 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-22 15:54:55,184 DEBUG [Listener at localhost/43289] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-22 15:54:55,186 INFO [Listener at localhost/43289] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-22 15:54:55,191 INFO [Listener at localhost/43289] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-22 15:54:55,191 INFO [Listener at localhost/43289] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-22 15:54:55,191 INFO [Listener at localhost/43289] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-22 15:54:55,196 INFO [Listener at localhost/43289] regionserver.HRegion(2745): Flushing f25498a0985673b023870241e50965da 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-22 15:54:55,618 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322145b10027b4741548d8470a5f5bebe5f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230322145b10027b4741548d8470a5f5bebe5f 2023-03-22 15:54:55,964 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-22 15:54:56,025 INFO [Listener at localhost/43289] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/.tmp/f1/0436b6f8e72b49d9809ec69370f4df79 2023-03-22 15:54:56,030 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/.tmp/f1/0436b6f8e72b49d9809ec69370f4df79 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79 2023-03-22 15:54:56,034 INFO [Listener at localhost/43289] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79, entries=1, sequenceid=5, filesize=4.9 K 2023-03-22 15:54:56,034 INFO [Listener at localhost/43289] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for f25498a0985673b023870241e50965da in 838ms, sequenceid=5, compaction requested=false 2023-03-22 15:54:56,035 DEBUG [Listener at localhost/43289] regionserver.HRegion(2446): Flush status journal for f25498a0985673b023870241e50965da: 2023-03-22 15:54:56,037 INFO [Listener at localhost/43289] regionserver.HRegion(2745): Flushing f25498a0985673b023870241e50965da 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-22 15:54:56,048 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303225103364798aa48ca9fb0f56ef2dea5e7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303225103364798aa48ca9fb0f56ef2dea5e7 2023-03-22 15:54:56,455 INFO [Listener at localhost/43289] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/.tmp/f1/34d752018a0f41a2a2a322332f6f02bd 2023-03-22 15:54:56,460 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/.tmp/f1/34d752018a0f41a2a2a322332f6f02bd as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/34d752018a0f41a2a2a322332f6f02bd 2023-03-22 15:54:56,464 INFO [Listener at localhost/43289] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/34d752018a0f41a2a2a322332f6f02bd, entries=1, sequenceid=9, filesize=4.9 K 2023-03-22 15:54:56,466 INFO [Listener at localhost/43289] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for f25498a0985673b023870241e50965da in 429ms, sequenceid=9, compaction requested=false 2023-03-22 15:54:56,466 DEBUG [Listener at localhost/43289] regionserver.HRegion(2446): Flush status journal for f25498a0985673b023870241e50965da: 2023-03-22 15:54:56,467 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-22 15:54:56,469 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-22 15:54:56,470 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-22 15:54:56,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:56,470 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501096470 2023-03-22 15:54:56,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-22 15:54:56,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-22 15:54:56,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-22 15:54:56,471 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x000915d9 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:54:56,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6aff4ed9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:54:56,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230322 for table='testGetAfterCompaction' and column='f1' 2023-03-22 15:54:56,519 INFO [Listener at localhost/43289] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679500496024; access_time=1679500495199; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-22 15:54:56,519 INFO [Listener at localhost/43289] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/34d752018a0f41a2a2a322332f6f02bd; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679500496455; access_time=1679500496039; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-22 15:54:56,530 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303227581d136a08343ab9afaea21307538f3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303227581d136a08343ab9afaea21307538f3 2023-03-22 15:54:56,530 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='testGetAfterCompaction' column='f1' 2023-03-22 15:54:56,544 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:54:56,545 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51046, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:54:56,551 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:54:56,553 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52370, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:54:56,563 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230322/f1/0e4b787c8f8f4435a0541fbeb307dd48 first=Optional[r1] last=Optional[r2] 2023-03-22 15:54:56,564 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{f1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230322/f1/0e4b787c8f8f4435a0541fbeb307dd48}] 2023-03-22 15:54:56,569 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230322/f1/0e4b787c8f8f4435a0541fbeb307dd48 for inclusion in f25498a0985673b023870241e50965da/f1 2023-03-22 15:54:56,570 INFO [Listener at localhost/43289] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679500496024; access_time=1679500495199; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-22 15:54:56,570 INFO [Listener at localhost/43289] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/34d752018a0f41a2a2a322332f6f02bd; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679500496455; access_time=1679500496039; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-22 15:54:56,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-22 15:54:56,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:54:56,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for f25498a0985673b023870241e50965da: 2023-03-22 15:54:56,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230322/f1/0e4b787c8f8f4435a0541fbeb307dd48 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testGetAfterCompaction__e8c18cvelfhpcc8741vbpb1ch0qn71u8e21v74lr4ve9glav0s6o0du263jfvvm9/f1/0e4b787c8f8f4435a0541fbeb307dd48 2023-03-22 15:54:56,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testGetAfterCompaction__e8c18cvelfhpcc8741vbpb1ch0qn71u8e21v74lr4ve9glav0s6o0du263jfvvm9/f1/0e4b787c8f8f4435a0541fbeb307dd48 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/1b5ad94170544ff2a1438dd28d802754_SeqId_12_ 2023-03-22 15:54:56,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testGetAfterCompaction__e8c18cvelfhpcc8741vbpb1ch0qn71u8e21v74lr4ve9glav0s6o0du263jfvvm9/f1/0e4b787c8f8f4435a0541fbeb307dd48 into f25498a0985673b023870241e50965da/f1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/1b5ad94170544ff2a1438dd28d802754_SeqId_12_ - updating store file list. 2023-03-22 15:54:56,581 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/1b5ad94170544ff2a1438dd28d802754_SeqId_12_ into f25498a0985673b023870241e50965da/f1 2023-03-22 15:54:56,581 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testGetAfterCompaction__e8c18cvelfhpcc8741vbpb1ch0qn71u8e21v74lr4ve9glav0s6o0du263jfvvm9/f1/0e4b787c8f8f4435a0541fbeb307dd48 into f25498a0985673b023870241e50965da/f1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/1b5ad94170544ff2a1438dd28d802754_SeqId_12_) 2023-03-22 15:54:56,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testGetAfterCompaction__e8c18cvelfhpcc8741vbpb1ch0qn71u8e21v74lr4ve9glav0s6o0du263jfvvm9/f1/0e4b787c8f8f4435a0541fbeb307dd48 2023-03-22 15:54:56,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): f25498a0985673b023870241e50965da/f1 is initiating minor compaction 2023-03-22 15:54:56,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@41f24b1f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:56,582 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f25498a0985673b023870241e50965da/f1 in testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:56,582 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/34d752018a0f41a2a2a322332f6f02bd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/1b5ad94170544ff2a1438dd28d802754_SeqId_12_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/.tmp, totalSize=10.0 K 2023-03-22 15:54:56,582 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 34d752018a0f41a2a2a322332f6f02bd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-22 15:54:56,582 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 1b5ad94170544ff2a1438dd28d802754_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-22 15:54:56,584 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='testGetAfterCompaction' column='f1' 2023-03-22 15:54:56,585 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:56,599 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230322145b10027b4741548d8470a5f5bebe5f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230322145b10027b4741548d8470a5f5bebe5f 2023-03-22 15:54:56,601 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303225103364798aa48ca9fb0f56ef2dea5e7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303225103364798aa48ca9fb0f56ef2dea5e7 2023-03-22 15:54:56,601 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-22 15:54:56,601 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230322' 2023-03-22 15:54:56,602 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): f25498a0985673b023870241e50965da#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:54:56,603 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230322 for table='testGetAfterCompaction' and column='f1' 2023-03-22 15:54:56,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-22 15:54:56,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:54:56,605 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-22 15:54:56,607 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 137 msec 2023-03-22 15:54:56,621 INFO [Listener at localhost/43289] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679500496024; access_time=1679500495199; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-22 15:54:56,621 INFO [Listener at localhost/43289] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/1b5ad94170544ff2a1438dd28d802754_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679500496526; access_time=1679500496505; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-22 15:54:56,621 INFO [Listener at localhost/43289] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/34d752018a0f41a2a2a322332f6f02bd; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679500496455; access_time=1679500496039; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-22 15:54:56,641 DEBUG [Listener at localhost/43289] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-22 15:54:56,642 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/.tmp/f1/a05989af39504b0ead970aaba739d0cc as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/a05989af39504b0ead970aaba739d0cc 2023-03-22 15:54:56,650 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51062, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-22 15:54:56,651 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38637] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:56,651 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in f25498a0985673b023870241e50965da/f1 of f25498a0985673b023870241e50965da into a05989af39504b0ead970aaba739d0cc(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:56,651 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38637] regionserver.HStore(1912): f25498a0985673b023870241e50965da/f1 is initiating minor compaction 2023-03-22 15:54:56,651 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f25498a0985673b023870241e50965da: 2023-03-22 15:54:56,651 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4db136e9; Because: User-triggered compaction for region testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. and family f1; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-22 15:54:56,651 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1679500496582; duration=0sec 2023-03-22 15:54:56,651 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegion(2259): Starting compaction of f25498a0985673b023870241e50965da/f1 in testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:56,651 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:56,652 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/a05989af39504b0ead970aaba739d0cc] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/.tmp, totalSize=10.0 K 2023-03-22 15:54:56,652 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting 0436b6f8e72b49d9809ec69370f4df79, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5 2023-03-22 15:54:56,652 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting a05989af39504b0ead970aaba739d0cc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-22 15:54:56,676 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] throttle.PressureAwareThroughputController(145): f25498a0985673b023870241e50965da#f1#compaction#77 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:54:56,696 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/34d752018a0f41a2a2a322332f6f02bd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/1b5ad94170544ff2a1438dd28d802754_SeqId_12_] to archive 2023-03-22 15:54:56,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:56,699 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/34d752018a0f41a2a2a322332f6f02bd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/34d752018a0f41a2a2a322332f6f02bd 2023-03-22 15:54:56,701 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/1b5ad94170544ff2a1438dd28d802754_SeqId_12_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/1b5ad94170544ff2a1438dd28d802754_SeqId_12_ 2023-03-22 15:54:56,704 INFO [Listener at localhost/43289] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679500496024; access_time=1679500495199; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-22 15:54:56,704 INFO [Listener at localhost/43289] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/a05989af39504b0ead970aaba739d0cc; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1679500496625; access_time=1679500496599; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-22 15:54:56,824 INFO [Listener at localhost/43289] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-22 15:54:56,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-22 15:54:56,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-22 15:54:56,839 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500496839"}]},"ts":"1679500496839"} 2023-03-22 15:54:56,841 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-22 15:54:56,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-22 15:54:56,842 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-22 15:54:56,844 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f25498a0985673b023870241e50965da, UNASSIGN}] 2023-03-22 15:54:56,845 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f25498a0985673b023870241e50965da, UNASSIGN 2023-03-22 15:54:56,845 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=f25498a0985673b023870241e50965da, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:56,846 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679500496845"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500496845"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500496845"}]},"ts":"1679500496845"} 2023-03-22 15:54:56,847 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure f25498a0985673b023870241e50965da, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:56,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-22 15:54:56,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f25498a0985673b023870241e50965da 2023-03-22 15:54:56,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f25498a0985673b023870241e50965da, disabling compactions & flushes 2023-03-22 15:54:57,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:57,113 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/.tmp/f1/e9fae30a95db4bbfa9073595c9c699a2 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/e9fae30a95db4bbfa9073595c9c699a2 2023-03-22 15:54:57,118 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in f25498a0985673b023870241e50965da/f1 of f25498a0985673b023870241e50965da into e9fae30a95db4bbfa9073595c9c699a2(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:54:57,118 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for f25498a0985673b023870241e50965da: 2023-03-22 15:54:57,118 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:57,118 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=1, time=1679500496651; duration=0sec 2023-03-22 15:54:57,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:57,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. after waiting 0 ms 2023-03-22 15:54:57,118 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:54:57,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:57,119 DEBUG [StoreCloser-testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/a05989af39504b0ead970aaba739d0cc] to archive 2023-03-22 15:54:57,119 DEBUG [StoreCloser-testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:54:57,121 DEBUG [StoreCloser-testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/0436b6f8e72b49d9809ec69370f4df79 2023-03-22 15:54:57,123 DEBUG [StoreCloser-testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/a05989af39504b0ead970aaba739d0cc to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/a05989af39504b0ead970aaba739d0cc 2023-03-22 15:54:57,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/recovered.edits/16.seqid, newMaxSeqId=16, maxSeqId=1 2023-03-22 15:54:57,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-22 15:54:57,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da. 2023-03-22 15:54:57,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f25498a0985673b023870241e50965da: 2023-03-22 15:54:57,130 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f25498a0985673b023870241e50965da 2023-03-22 15:54:57,130 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=f25498a0985673b023870241e50965da, regionState=CLOSED 2023-03-22 15:54:57,130 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679500497130"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500497130"}]},"ts":"1679500497130"} 2023-03-22 15:54:57,132 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-22 15:54:57,132 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure f25498a0985673b023870241e50965da, server=jenkins-hbase4.apache.org,38637,1679500416185 in 284 msec 2023-03-22 15:54:57,134 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-22 15:54:57,134 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=f25498a0985673b023870241e50965da, UNASSIGN in 288 msec 2023-03-22 15:54:57,134 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500497134"}]},"ts":"1679500497134"} 2023-03-22 15:54:57,135 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-22 15:54:57,138 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-22 15:54:57,140 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 307 msec 2023-03-22 15:54:57,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-22 15:54:57,143 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-22 15:54:57,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-22 15:54:57,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-22 15:54:57,157 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-22 15:54:57,159 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-22 15:54:57,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-22 15:54:57,163 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da 2023-03-22 15:54:57,171 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1, FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/recovered.edits] 2023-03-22 15:54:57,176 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/e9fae30a95db4bbfa9073595c9c699a2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/f1/e9fae30a95db4bbfa9073595c9c699a2 2023-03-22 15:54:57,179 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/recovered.edits/16.seqid to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da/recovered.edits/16.seqid 2023-03-22 15:54:57,180 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testGetAfterCompaction/f25498a0985673b023870241e50965da 2023-03-22 15:54:57,180 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-22 15:54:57,181 DEBUG [PEWorker-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-22 15:54:57,181 DEBUG [PEWorker-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-22 15:54:57,185 DEBUG [PEWorker-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303227581d136a08343ab9afaea21307538f3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303227581d136a08343ab9afaea21307538f3 2023-03-22 15:54:57,186 DEBUG [PEWorker-1] backup.HFileArchiver(596): Deleted hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-22 15:54:57,188 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-22 15:54:57,216 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-22 15:54:57,227 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-22 15:54:57,228 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-22 15:54:57,228 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-22 15:54:57,229 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679500497228"}]},"ts":"9223372036854775807"} 2023-03-22 15:54:57,231 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-22 15:54:57,231 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f25498a0985673b023870241e50965da, NAME => 'testGetAfterCompaction,,1679500494073.f25498a0985673b023870241e50965da.', STARTKEY => '', ENDKEY => ''}] 2023-03-22 15:54:57,231 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-22 15:54:57,231 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1679500497231"}]},"ts":"9223372036854775807"} 2023-03-22 15:54:57,233 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-22 15:54:57,238 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-22 15:54:57,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 89 msec 2023-03-22 15:54:57,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-22 15:54:57,261 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-22 15:54:57,279 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=261 (was 252) - Thread LEAK? -, OpenFileDescriptor=644 (was 635) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=403 (was 394) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6479 (was 6478) - AvailableMemoryMB LEAK? - 2023-03-22 15:54:57,296 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=261, OpenFileDescriptor=644, MaxFileDescriptor=60000, SystemLoadAverage=403, ProcessCount=170, AvailableMemoryMB=6478 2023-03-22 15:54:57,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-22 15:54:57,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-22 15:54:57,301 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:54:57,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-22 15:54:57,301 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:54:57,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-22 15:54:57,304 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:57,304 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:57,305 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3 empty. 2023-03-22 15:54:57,308 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725 empty. 2023-03-22 15:54:57,308 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:57,308 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:57,308 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:57,309 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400 empty. 2023-03-22 15:54:57,309 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:57,309 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-22 15:54:57,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-22 15:54:57,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-22 15:54:57,728 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-22 15:54:57,729 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2345ee45559d48c9feb8a4a60984e2c3, NAME => 'testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:57,730 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => fc08f2f22b58b2bb5e4b114f03a4e725, NAME => 'testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:57,730 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => b3544ae53f253dc547c7d54a0e776400, NAME => 'testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:54:57,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-22 15:54:57,972 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing b3544ae53f253dc547c7d54a0e776400, disabling compactions & flushes 2023-03-22 15:54:58,139 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. after waiting 0 ms 2023-03-22 15:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:54:58,139 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:54:58,146 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:58,146 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing fc08f2f22b58b2bb5e4b114f03a4e725, disabling compactions & flushes 2023-03-22 15:54:58,146 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:54:58,146 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:54:58,146 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. after waiting 0 ms 2023-03-22 15:54:58,146 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:54:58,146 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:54:58,146 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:54:58,147 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:58,147 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 2345ee45559d48c9feb8a4a60984e2c3, disabling compactions & flushes 2023-03-22 15:54:58,147 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:54:58,147 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:54:58,147 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. after waiting 0 ms 2023-03-22 15:54:58,147 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:54:58,147 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:54:58,147 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:54:58,150 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:54:58,151 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500498151"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500498151"}]},"ts":"1679500498151"} 2023-03-22 15:54:58,151 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500498151"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500498151"}]},"ts":"1679500498151"} 2023-03-22 15:54:58,151 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500498151"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500498151"}]},"ts":"1679500498151"} 2023-03-22 15:54:58,154 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-22 15:54:58,154 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:54:58,155 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500498155"}]},"ts":"1679500498155"} 2023-03-22 15:54:58,156 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-22 15:54:58,160 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, ASSIGN}] 2023-03-22 15:54:58,162 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, ASSIGN 2023-03-22 15:54:58,162 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, ASSIGN 2023-03-22 15:54:58,163 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, ASSIGN 2023-03-22 15:54:58,163 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:58,164 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:58,164 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:54:58,315 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=2345ee45559d48c9feb8a4a60984e2c3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:58,315 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=fc08f2f22b58b2bb5e4b114f03a4e725, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:58,315 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500498315"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500498315"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500498315"}]},"ts":"1679500498315"} 2023-03-22 15:54:58,315 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=b3544ae53f253dc547c7d54a0e776400, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:58,315 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500498315"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500498315"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500498315"}]},"ts":"1679500498315"} 2023-03-22 15:54:58,316 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500498315"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500498315"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500498315"}]},"ts":"1679500498315"} 2023-03-22 15:54:58,317 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 2345ee45559d48c9feb8a4a60984e2c3, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:58,319 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure fc08f2f22b58b2bb5e4b114f03a4e725, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:58,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure b3544ae53f253dc547c7d54a0e776400, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:54:58,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-22 15:54:58,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:54:58,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fc08f2f22b58b2bb5e4b114f03a4e725, NAME => 'testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.', STARTKEY => '1', ENDKEY => '2'} 2023-03-22 15:54:58,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:58,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:58,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:58,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:58,474 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:58,475 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1 2023-03-22 15:54:58,476 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1 2023-03-22 15:54:58,476 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fc08f2f22b58b2bb5e4b114f03a4e725 columnFamilyName family1 2023-03-22 15:54:58,481 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(310): Store=fc08f2f22b58b2bb5e4b114f03a4e725/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:58,482 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:58,483 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2 2023-03-22 15:54:58,483 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2 2023-03-22 15:54:58,483 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fc08f2f22b58b2bb5e4b114f03a4e725 columnFamilyName family2 2023-03-22 15:54:58,485 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(310): Store=fc08f2f22b58b2bb5e4b114f03a4e725/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:58,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:58,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:58,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:54:58,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:54:58,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:58,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fc08f2f22b58b2bb5e4b114f03a4e725; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68560173, jitterRate=0.021626189351081848}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:54:58,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:54:58,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725., pid=84, masterSystemTime=1679500498469 2023-03-22 15:54:58,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:54:58,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:54:58,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:54:58,499 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=fc08f2f22b58b2bb5e4b114f03a4e725, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:58,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b3544ae53f253dc547c7d54a0e776400, NAME => 'testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.', STARTKEY => '2', ENDKEY => ''} 2023-03-22 15:54:58,500 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500498499"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500498499"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500498499"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500498499"}]},"ts":"1679500498499"} 2023-03-22 15:54:58,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:58,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:58,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:58,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:58,503 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:58,504 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1 2023-03-22 15:54:58,504 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1 2023-03-22 15:54:58,505 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3544ae53f253dc547c7d54a0e776400 columnFamilyName family1 2023-03-22 15:54:58,506 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(310): Store=b3544ae53f253dc547c7d54a0e776400/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:58,507 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:58,508 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-03-22 15:54:58,508 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure fc08f2f22b58b2bb5e4b114f03a4e725, server=jenkins-hbase4.apache.org,38637,1679500416185 in 186 msec 2023-03-22 15:54:58,509 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, ASSIGN in 349 msec 2023-03-22 15:54:58,510 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2 2023-03-22 15:54:58,510 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2 2023-03-22 15:54:58,510 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3544ae53f253dc547c7d54a0e776400 columnFamilyName family2 2023-03-22 15:54:58,511 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(310): Store=b3544ae53f253dc547c7d54a0e776400/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:58,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:58,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:58,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:54:58,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:54:58,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:58,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b3544ae53f253dc547c7d54a0e776400; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67783262, jitterRate=0.010049313306808472}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:54:58,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:54:58,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400., pid=85, masterSystemTime=1679500498469 2023-03-22 15:54:58,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:54:58,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:54:58,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:54:58,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2345ee45559d48c9feb8a4a60984e2c3, NAME => 'testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.', STARTKEY => '', ENDKEY => '1'} 2023-03-22 15:54:58,523 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=b3544ae53f253dc547c7d54a0e776400, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:58,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:58,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:54:58,524 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500498523"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500498523"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500498523"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500498523"}]},"ts":"1679500498523"} 2023-03-22 15:54:58,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:58,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:58,527 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=82 2023-03-22 15:54:58,527 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure b3544ae53f253dc547c7d54a0e776400, server=jenkins-hbase4.apache.org,38637,1679500416185 in 205 msec 2023-03-22 15:54:58,529 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, ASSIGN in 368 msec 2023-03-22 15:54:58,535 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:58,537 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1 2023-03-22 15:54:58,537 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1 2023-03-22 15:54:58,538 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2345ee45559d48c9feb8a4a60984e2c3 columnFamilyName family1 2023-03-22 15:54:58,539 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(310): Store=2345ee45559d48c9feb8a4a60984e2c3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:58,539 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:58,540 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2 2023-03-22 15:54:58,540 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2 2023-03-22 15:54:58,540 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2345ee45559d48c9feb8a4a60984e2c3 columnFamilyName family2 2023-03-22 15:54:58,541 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(310): Store=2345ee45559d48c9feb8a4a60984e2c3/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:54:58,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:58,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:58,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:54:58,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:54:58,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:54:58,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2345ee45559d48c9feb8a4a60984e2c3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63947914, jitterRate=-0.04710182547569275}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:54:58,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:54:58,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3., pid=83, masterSystemTime=1679500498469 2023-03-22 15:54:58,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:54:58,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:54:58,550 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=2345ee45559d48c9feb8a4a60984e2c3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:58,550 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500498550"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500498550"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500498550"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500498550"}]},"ts":"1679500498550"} 2023-03-22 15:54:58,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-03-22 15:54:58,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 2345ee45559d48c9feb8a4a60984e2c3, server=jenkins-hbase4.apache.org,38637,1679500416185 in 234 msec 2023-03-22 15:54:58,554 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-03-22 15:54:58,555 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, ASSIGN in 394 msec 2023-03-22 15:54:58,555 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:54:58,555 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500498555"}]},"ts":"1679500498555"} 2023-03-22 15:54:58,556 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-22 15:54:58,558 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:54:58,559 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2600 sec 2023-03-22 15:54:59,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-22 15:54:59,408 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-22 15:54:59,417 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:54:59,419 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:54:59,420 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:54:59,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:54:59,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-22 15:54:59,428 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-22 15:54:59,428 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:59,428 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-22 15:54:59,428 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-22 15:54:59,428 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:54:59,428 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:54:59,430 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:59,430 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:54:59,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:54:59,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:54:59,430 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:54:59,430 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:54:59,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:54:59,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:54:59,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:54:59,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:54:59,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-22 15:54:59,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-22 15:54:59,435 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-22 15:54:59,435 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:54:59,435 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-22 15:54:59,435 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:54:59,435 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. started... 2023-03-22 15:54:59,436 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fc08f2f22b58b2bb5e4b114f03a4e725 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-22 15:54:59,437 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:54:59,437 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:54:59,437 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:54:59,437 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. started... 2023-03-22 15:54:59,437 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. started... 2023-03-22 15:54:59,437 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:54:59,437 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2345ee45559d48c9feb8a4a60984e2c3 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-22 15:54:59,437 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3544ae53f253dc547c7d54a0e776400 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-22 15:54:59,453 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032262ef361c18a54c10b19099f1a899c95e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032262ef361c18a54c10b19099f1a899c95e 2023-03-22 15:54:59,460 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/f9075a905dd8452c8f347a85a0de825f 2023-03-22 15:54:59,854 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303227142606f287940e88cf21ec6c6677b0e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303227142606f287940e88cf21ec6c6677b0e 2023-03-22 15:54:59,854 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322c616bcebc940418b8c1df3a672466bc6 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322c616bcebc940418b8c1df3a672466bc6 2023-03-22 15:54:59,872 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032276687dc503704c4393bed8418c7e7461 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032276687dc503704c4393bed8418c7e7461 2023-03-22 15:55:00,259 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/7917639753e84614aefeb4c0d38d5c9d 2023-03-22 15:55:00,259 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/ce9c7293befc4ac2b83081d67235637f 2023-03-22 15:55:00,276 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/7abb4a6c3d5d43a894a2c727b03c262a 2023-03-22 15:55:00,280 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/f9075a905dd8452c8f347a85a0de825f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/f9075a905dd8452c8f347a85a0de825f 2023-03-22 15:55:00,283 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/f9075a905dd8452c8f347a85a0de825f, entries=4, sequenceid=5, filesize=5.4 K 2023-03-22 15:55:00,283 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/7abb4a6c3d5d43a894a2c727b03c262a as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7abb4a6c3d5d43a894a2c727b03c262a 2023-03-22 15:55:00,286 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7abb4a6c3d5d43a894a2c727b03c262a, entries=2, sequenceid=5, filesize=5.1 K 2023-03-22 15:55:00,287 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b3544ae53f253dc547c7d54a0e776400 in 850ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:00,287 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-22 15:55:00,288 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:00,288 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:00,288 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:00,674 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322d73886fbadb84804a50fc3023579319b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322d73886fbadb84804a50fc3023579319b 2023-03-22 15:55:00,674 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303229a9bc402608441c48b9f5dbc065f00f1 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303229a9bc402608441c48b9f5dbc065f00f1 2023-03-22 15:55:01,080 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/71885845d5394c4f8c6fa2cf66aee993 2023-03-22 15:55:01,080 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/2aa4237f0ab246f69abbc12410b88bba 2023-03-22 15:55:01,084 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/7917639753e84614aefeb4c0d38d5c9d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/7917639753e84614aefeb4c0d38d5c9d 2023-03-22 15:55:01,084 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/ce9c7293befc4ac2b83081d67235637f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/ce9c7293befc4ac2b83081d67235637f 2023-03-22 15:55:01,088 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/ce9c7293befc4ac2b83081d67235637f, entries=4, sequenceid=5, filesize=5.4 K 2023-03-22 15:55:01,088 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/7917639753e84614aefeb4c0d38d5c9d, entries=4, sequenceid=5, filesize=5.4 K 2023-03-22 15:55:01,088 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/71885845d5394c4f8c6fa2cf66aee993 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/71885845d5394c4f8c6fa2cf66aee993 2023-03-22 15:55:01,088 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/2aa4237f0ab246f69abbc12410b88bba as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/2aa4237f0ab246f69abbc12410b88bba 2023-03-22 15:55:01,091 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/71885845d5394c4f8c6fa2cf66aee993, entries=2, sequenceid=5, filesize=5.1 K 2023-03-22 15:55:01,092 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/2aa4237f0ab246f69abbc12410b88bba, entries=2, sequenceid=5, filesize=5.1 K 2023-03-22 15:55:01,092 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 2345ee45559d48c9feb8a4a60984e2c3 in 1655ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:01,093 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:01,093 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:01,093 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:01,093 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for fc08f2f22b58b2bb5e4b114f03a4e725 in 1657ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:01,093 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:01,093 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:01,093 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:01,093 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:01,093 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,093 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-22 15:55:01,093 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-22 15:55:01,095 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,095 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:01,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:01,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:01,095 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:01,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:01,095 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:01,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:01,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:01,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:01,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-22 15:55:01,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22b198bb[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:01,097 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-22 15:55:01,097 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:01,098 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:01,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:01,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:01,098 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-22 15:55:01,098 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-22 15:55:01,098 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:01,098 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,098 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:01,101 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,101 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-22 15:55:01,101 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:01,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:01,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:01,101 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-22 15:55:01,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:01,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:01,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:01,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:01,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:01,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:01,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-22 15:55:01,102 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-22 15:55:01,103 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:01,103 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:01,103 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-22 15:55:01,103 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:01,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:01,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:01,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:01,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:01,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:01,105 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:01,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:01,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:01,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:01,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:01,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:01,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:01,105 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:01,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:01,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,108 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:01,108 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,108 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:01,108 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:01,108 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:01,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-22 15:55:01,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:01,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:01,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:01,108 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:01,109 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,109 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:01,109 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:01,109 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:01,109 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:01,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:01,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:01,109 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:01,209 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:01,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:01,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:01,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:01,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:01,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:01,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:01,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-22 15:55:01,215 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-22 15:55:01,215 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:01,215 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-22 15:55:01,215 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-22 15:55:01,215 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:01,215 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:01,217 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,217 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:01,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:01,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:01,217 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:01,217 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:01,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:01,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:01,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:01,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:01,218 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-22 15:55:01,218 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-22 15:55:01,218 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-22 15:55:01,218 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-22 15:55:01,218 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:01,218 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-22 15:55:01,218 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:01,219 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. started... 2023-03-22 15:55:01,219 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:01,219 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. started... 2023-03-22 15:55:01,219 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fc08f2f22b58b2bb5e4b114f03a4e725 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-22 15:55:01,219 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3544ae53f253dc547c7d54a0e776400 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-22 15:55:01,223 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:01,223 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:01,223 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:01,223 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. started... 2023-03-22 15:55:01,223 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2345ee45559d48c9feb8a4a60984e2c3 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-22 15:55:01,249 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322059296d521f74be3822284aae2c125ab to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322059296d521f74be3822284aae2c125ab 2023-03-22 15:55:01,644 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322e5039a9c9c6e462ea98ef70511033233 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322e5039a9c9c6e462ea98ef70511033233 2023-03-22 15:55:01,646 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303229ff89dda9c844b89bb8dd67fe572459c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303229ff89dda9c844b89bb8dd67fe572459c 2023-03-22 15:55:01,649 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/03e5d7bf4bbc4f9dba04343703393d6b 2023-03-22 15:55:01,650 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/f3b4e4f5037842a393873a3f040dc476 2023-03-22 15:55:01,654 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/b8feeb8b3e8c416882ec0c124ebd0f4d 2023-03-22 15:55:01,665 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322cca9738e1f2b4157bec9d6e75deb0e10 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322cca9738e1f2b4157bec9d6e75deb0e10 2023-03-22 15:55:02,069 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322af819ee46ca44f64b03db21e05ff8841 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322af819ee46ca44f64b03db21e05ff8841 2023-03-22 15:55:02,072 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/a17d54ef6c504ff5aa86672257e08710 2023-03-22 15:55:02,073 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303224e85c149281a40baac2d2780fe54cc0f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303224e85c149281a40baac2d2780fe54cc0f 2023-03-22 15:55:02,077 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/f3b4e4f5037842a393873a3f040dc476 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/f3b4e4f5037842a393873a3f040dc476 2023-03-22 15:55:02,080 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/f3b4e4f5037842a393873a3f040dc476, entries=4, sequenceid=9, filesize=5.4 K 2023-03-22 15:55:02,083 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/a17d54ef6c504ff5aa86672257e08710 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/a17d54ef6c504ff5aa86672257e08710 2023-03-22 15:55:02,086 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/a17d54ef6c504ff5aa86672257e08710, entries=2, sequenceid=9, filesize=5.1 K 2023-03-22 15:55:02,087 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 2345ee45559d48c9feb8a4a60984e2c3 in 864ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:02,087 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:02,087 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:02,087 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:02,475 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/70e5363498bd448089cef77631a32139 2023-03-22 15:55:02,479 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/b8feeb8b3e8c416882ec0c124ebd0f4d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/b8feeb8b3e8c416882ec0c124ebd0f4d 2023-03-22 15:55:02,483 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/b8feeb8b3e8c416882ec0c124ebd0f4d, entries=4, sequenceid=9, filesize=5.4 K 2023-03-22 15:55:02,483 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/4dd66f50257149a69fc3e0617e774e1c 2023-03-22 15:55:02,483 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/70e5363498bd448089cef77631a32139 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/70e5363498bd448089cef77631a32139 2023-03-22 15:55:02,487 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/70e5363498bd448089cef77631a32139, entries=2, sequenceid=9, filesize=5.1 K 2023-03-22 15:55:02,487 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/03e5d7bf4bbc4f9dba04343703393d6b as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/03e5d7bf4bbc4f9dba04343703393d6b 2023-03-22 15:55:02,488 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b3544ae53f253dc547c7d54a0e776400 in 1269ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:02,488 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:02,488 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:02,488 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:02,490 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/03e5d7bf4bbc4f9dba04343703393d6b, entries=4, sequenceid=9, filesize=5.4 K 2023-03-22 15:55:02,491 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/4dd66f50257149a69fc3e0617e774e1c as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/4dd66f50257149a69fc3e0617e774e1c 2023-03-22 15:55:02,494 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/4dd66f50257149a69fc3e0617e774e1c, entries=2, sequenceid=9, filesize=5.1 K 2023-03-22 15:55:02,494 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for fc08f2f22b58b2bb5e4b114f03a4e725 in 1275ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:02,494 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:02,494 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:02,494 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:02,495 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:02,495 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,495 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-22 15:55:02,495 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-22 15:55:02,500 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,500 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:02,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:02,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:02,500 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:02,500 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:02,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:02,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:02,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:02,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:02,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-22 15:55:02,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2096d6fd[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:02,502 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-22 15:55:02,502 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:02,503 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:02,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:02,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:02,503 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-22 15:55:02,503 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-22 15:55:02,503 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:02,503 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,503 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:02,505 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-22 15:55:02,505 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:02,505 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,505 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-22 15:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:02,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:02,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:02,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:02,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:02,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-22 15:55:02,507 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-22 15:55:02,507 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:02,507 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:02,507 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-22 15:55:02,507 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:02,508 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:02,508 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:02,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:02,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:02,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:02,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:02,508 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:02,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:02,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:02,509 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:02,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:02,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:02,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:02,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:02,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:02,510 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:02,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:02,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:02,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:02,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-22 15:55:02,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:02,513 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:02,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:02,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:02,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:02,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:02,513 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:02,613 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:02,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:02,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:02,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:02,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:02,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:02,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:02,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-22 15:55:02,620 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-22 15:55:02,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:02,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-22 15:55:02,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-22 15:55:02,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:02,620 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:02,622 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,622 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:02,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:02,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:02,622 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:02,622 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:02,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:02,622 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:02,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:02,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:02,623 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-22 15:55:02,623 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-22 15:55:02,623 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-22 15:55:02,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-22 15:55:02,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:02,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-22 15:55:02,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:02,623 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:02,623 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:02,623 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:02,623 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. started... 2023-03-22 15:55:02,623 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:02,623 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. started... 2023-03-22 15:55:02,623 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. started... 2023-03-22 15:55:02,623 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fc08f2f22b58b2bb5e4b114f03a4e725 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-22 15:55:02,624 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3544ae53f253dc547c7d54a0e776400 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-22 15:55:02,624 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2345ee45559d48c9feb8a4a60984e2c3 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-22 15:55:03,044 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322b5802331476e4c4d8d93c74275b69e33 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322b5802331476e4c4d8d93c74275b69e33 2023-03-22 15:55:03,044 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303220495f367c8bf4559a0ce2059d3c42779 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303220495f367c8bf4559a0ce2059d3c42779 2023-03-22 15:55:03,044 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322127ec934bcb24f44b0fdfe9f5c430400 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322127ec934bcb24f44b0fdfe9f5c430400 2023-03-22 15:55:03,056 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/a0ad7a2265a043bc80f148a22e45cb6f 2023-03-22 15:55:03,454 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/043c5cd69fe94f4bb70b8fbff9329fee 2023-03-22 15:55:03,457 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/7876950a8aaf4d5eacc03c89aba69af3 2023-03-22 15:55:03,468 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303222668112f32ab495cb8a4b79b50260eca to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303222668112f32ab495cb8a4b79b50260eca 2023-03-22 15:55:03,868 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322320558028cbb41f4a7071f12aabcf0bd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322320558028cbb41f4a7071f12aabcf0bd 2023-03-22 15:55:03,869 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322dd442995df4a42ab96923a9fcf01df24 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322dd442995df4a42ab96923a9fcf01df24 2023-03-22 15:55:03,873 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/8b37dea4807341f996a5efdd042112a0 2023-03-22 15:55:03,879 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/a0ad7a2265a043bc80f148a22e45cb6f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/a0ad7a2265a043bc80f148a22e45cb6f 2023-03-22 15:55:03,883 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/a0ad7a2265a043bc80f148a22e45cb6f, entries=4, sequenceid=13, filesize=5.4 K 2023-03-22 15:55:03,883 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/8b37dea4807341f996a5efdd042112a0 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/8b37dea4807341f996a5efdd042112a0 2023-03-22 15:55:03,886 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/8b37dea4807341f996a5efdd042112a0, entries=2, sequenceid=13, filesize=5.1 K 2023-03-22 15:55:03,887 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for fc08f2f22b58b2bb5e4b114f03a4e725 in 1264ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:03,887 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:03,887 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:03,887 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:03,974 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:55:04,277 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/ae4e7e46dd414d1794011c71e8b2d40d 2023-03-22 15:55:04,277 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/c9aaf840692146c29c916b1885331b30 2023-03-22 15:55:04,281 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/7876950a8aaf4d5eacc03c89aba69af3 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/7876950a8aaf4d5eacc03c89aba69af3 2023-03-22 15:55:04,281 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/043c5cd69fe94f4bb70b8fbff9329fee as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/043c5cd69fe94f4bb70b8fbff9329fee 2023-03-22 15:55:04,285 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/7876950a8aaf4d5eacc03c89aba69af3, entries=4, sequenceid=13, filesize=5.4 K 2023-03-22 15:55:04,285 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/043c5cd69fe94f4bb70b8fbff9329fee, entries=4, sequenceid=13, filesize=5.4 K 2023-03-22 15:55:04,286 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/ae4e7e46dd414d1794011c71e8b2d40d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/ae4e7e46dd414d1794011c71e8b2d40d 2023-03-22 15:55:04,286 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/c9aaf840692146c29c916b1885331b30 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/c9aaf840692146c29c916b1885331b30 2023-03-22 15:55:04,290 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/ae4e7e46dd414d1794011c71e8b2d40d, entries=2, sequenceid=13, filesize=5.1 K 2023-03-22 15:55:04,290 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/c9aaf840692146c29c916b1885331b30, entries=2, sequenceid=13, filesize=5.1 K 2023-03-22 15:55:04,291 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 2345ee45559d48c9feb8a4a60984e2c3 in 1668ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:04,291 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b3544ae53f253dc547c7d54a0e776400 in 1667ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:04,291 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:04,291 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:04,291 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:04,292 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:04,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:04,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:04,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:04,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-22 15:55:04,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-22 15:55:04,293 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:04,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:04,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:04,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:04,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-22 15:55:04,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14d864fb[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:04,295 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-22 15:55:04,295 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:04,296 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:04,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:04,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:04,297 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-22 15:55:04,297 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-22 15:55:04,297 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:04,297 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,297 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:04,298 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-22 15:55:04,298 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,298 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:04,298 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-22 15:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:04,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:04,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:04,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:04,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:04,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:04,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-22 15:55:04,300 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-22 15:55:04,300 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:04,300 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:04,300 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-22 15:55:04,300 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:04,303 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:04,303 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:04,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:04,303 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:04,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:04,303 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:04,304 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:04,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:04,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,310 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,310 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:04,310 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:04,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-22 15:55:04,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-22 15:55:04,310 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:04,310 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:04,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:04,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:04,310 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:04,310 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:04,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:04,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:04,311 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:04,311 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:04,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:04,411 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:04,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:04,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:04,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:04,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:04,415 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:04,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:04,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-22 15:55:04,420 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-22 15:55:04,420 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:04,420 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-22 15:55:04,420 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-22 15:55:04,420 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:04,420 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:04,422 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:04,422 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:04,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:04,422 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:04,422 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:04,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:04,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:04,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:04,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:04,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-22 15:55:04,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-22 15:55:04,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-22 15:55:04,423 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-22 15:55:04,423 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:04,423 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-22 15:55:04,424 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:04,424 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:04,424 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. started... 2023-03-22 15:55:04,424 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:04,424 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. started... 2023-03-22 15:55:04,424 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:04,424 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fc08f2f22b58b2bb5e4b114f03a4e725 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-22 15:55:04,424 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:04,424 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. started... 2023-03-22 15:55:04,424 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2345ee45559d48c9feb8a4a60984e2c3 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-22 15:55:04,424 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3544ae53f253dc547c7d54a0e776400 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-22 15:55:04,439 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303227dae9615fad648e6b55bc956feb5a1e4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303227dae9615fad648e6b55bc956feb5a1e4 2023-03-22 15:55:04,840 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032292092a0688b74b8292c4011b93e1193a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032292092a0688b74b8292c4011b93e1193a 2023-03-22 15:55:04,840 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322f18742089e924f019e0769cd09b681ed to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322f18742089e924f019e0769cd09b681ed 2023-03-22 15:55:04,844 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/19d38f1e5b9a417fbe004d4965abcf7c 2023-03-22 15:55:04,858 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032208d494d999aa4eaba06d6b4130ed46f2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032208d494d999aa4eaba06d6b4130ed46f2 2023-03-22 15:55:05,105 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-22 15:55:05,245 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/16f191502d2a4ad2bd45864da26c4836 2023-03-22 15:55:05,247 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/8a275a5fcb784f13804cf407ba7a368d 2023-03-22 15:55:05,262 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/18e0114e20734d5c87b318ace88eb8a2 2023-03-22 15:55:05,266 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/19d38f1e5b9a417fbe004d4965abcf7c as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/19d38f1e5b9a417fbe004d4965abcf7c 2023-03-22 15:55:05,290 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:55:05,290 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x317f0b25 to 127.0.0.1:58225 2023-03-22 15:55:05,290 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:05,291 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:55:05,291 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x360e9f25 to 127.0.0.1:58225 2023-03-22 15:55:05,291 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:05,291 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:55:05,291 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3661ce42 to 127.0.0.1:58225 2023-03-22 15:55:05,291 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:05,291 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1485a1f1 to 127.0.0.1:58225 2023-03-22 15:55:05,291 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:05,291 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x61c0fd3a to 127.0.0.1:58225 2023-03-22 15:55:05,291 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:05,291 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:55:05,291 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x18c333b6 to 127.0.0.1:58225 2023-03-22 15:55:05,291 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:05,292 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:55:05,292 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x000915d9 to 127.0.0.1:58225 2023-03-22 15:55:05,292 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:05,295 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/19d38f1e5b9a417fbe004d4965abcf7c, entries=4, sequenceid=17, filesize=5.4 K 2023-03-22 15:55:05,295 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/18e0114e20734d5c87b318ace88eb8a2 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/18e0114e20734d5c87b318ace88eb8a2 2023-03-22 15:55:05,298 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/18e0114e20734d5c87b318ace88eb8a2, entries=2, sequenceid=17, filesize=5.1 K 2023-03-22 15:55:05,299 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b3544ae53f253dc547c7d54a0e776400 in 875ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:05,299 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:05,299 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:05,299 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:05,670 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303227b9d7305868c40e5a8ffcffeaf6487ad to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303227b9d7305868c40e5a8ffcffeaf6487ad 2023-03-22 15:55:05,673 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032266fce2129b984cbeaf78d4f2e1977b69 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032266fce2129b984cbeaf78d4f2e1977b69 2023-03-22 15:55:05,685 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/a5af574092a54fc19d8d96cf59f9ec91 2023-03-22 15:55:05,690 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/8a275a5fcb784f13804cf407ba7a368d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/8a275a5fcb784f13804cf407ba7a368d 2023-03-22 15:55:05,696 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/8a275a5fcb784f13804cf407ba7a368d, entries=4, sequenceid=17, filesize=5.4 K 2023-03-22 15:55:05,696 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/a5af574092a54fc19d8d96cf59f9ec91 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/a5af574092a54fc19d8d96cf59f9ec91 2023-03-22 15:55:05,700 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/a5af574092a54fc19d8d96cf59f9ec91, entries=2, sequenceid=17, filesize=5.1 K 2023-03-22 15:55:05,701 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for fc08f2f22b58b2bb5e4b114f03a4e725 in 1277ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:05,701 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:05,701 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:05,701 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:06,078 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/9f3ebcd7510d453d968b5291f038a60a 2023-03-22 15:55:06,083 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/16f191502d2a4ad2bd45864da26c4836 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/16f191502d2a4ad2bd45864da26c4836 2023-03-22 15:55:06,089 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/16f191502d2a4ad2bd45864da26c4836, entries=4, sequenceid=17, filesize=5.4 K 2023-03-22 15:55:06,090 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/9f3ebcd7510d453d968b5291f038a60a as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/9f3ebcd7510d453d968b5291f038a60a 2023-03-22 15:55:06,094 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/9f3ebcd7510d453d968b5291f038a60a, entries=2, sequenceid=17, filesize=5.1 K 2023-03-22 15:55:06,095 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 2345ee45559d48c9feb8a4a60984e2c3 in 1671ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:06,095 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:06,095 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:06,095 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:06,095 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:06,095 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,096 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-22 15:55:06,096 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-22 15:55:06,097 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,097 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:06,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:06,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:06,098 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:06,098 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:06,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:06,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:06,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-22 15:55:06,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4dc0154c[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:06,101 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-22 15:55:06,102 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:06,103 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:06,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:06,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:06,103 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-22 15:55:06,103 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-22 15:55:06,104 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:06,104 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,104 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:06,112 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,112 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-22 15:55:06,112 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:06,112 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-22 15:55:06,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:06,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:06,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:06,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:06,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:06,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:06,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:06,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:06,115 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-22 15:55:06,115 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:06,115 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:06,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-22 15:55:06,115 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-22 15:55:06,115 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:06,117 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:06,117 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:06,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:06,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:06,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:06,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:06,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:06,117 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:06,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:06,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:06,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:06,118 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:06,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:06,121 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:06,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:06,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,124 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,124 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:06,124 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:06,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:06,124 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:06,124 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:06,124 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:06,124 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,124 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:06,124 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:06,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:06,124 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:06,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-22 15:55:06,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:06,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-22 15:55:06,125 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:06,125 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:06,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:06,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:06,225 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:06,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:06,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-22 15:55:06,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1679500506240 (current time:1679500506240). 2023-03-22 15:55:06,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-22 15:55:06,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] zookeeper.ReadOnlyZKClient(139): Connect 0x4ec7a068 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:55:06,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1dc9cae0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:55:06,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:55:06,254 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60070, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:55:06,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4ec7a068 to 127.0.0.1:58225 2023-03-22 15:55:06,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:06,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-22 15:55:06,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-22 15:55:06,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-22 15:55:06,268 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-22 15:55:06,268 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1679501106268 2023-03-22 15:55:06,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-22 15:55:06,269 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1679500506226 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-22 15:55:06,273 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-22 15:55:06,273 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-22 15:55:06,274 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 6 msec 2023-03-22 15:55:06,274 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-22 15:55:06,275 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1679501106275 2023-03-22 15:55:06,284 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:06,284 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-22 15:55:06,288 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1679500506226 2023-03-22 15:55:06,300 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1679500506226' 2023-03-22 15:55:06,300 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-22 15:55:06,300 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1679500506226' starting 'acquire' 2023-03-22 15:55:06,300 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1679500506226', kicking off acquire phase on members. 2023-03-22 15:55:06,360 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1679500506226 2023-03-22 15:55:06,360 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1679500506226 2023-03-22 15:55:06,362 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-22 15:55:06,362 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-22 15:55:06,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-22 15:55:06,362 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,363 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:06,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1679500506226 2023-03-22 15:55:06,363 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1679500506226 2023-03-22 15:55:06,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-22 15:55:06,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1679500506226 2023-03-22 15:55:06,364 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1679500506226 from table testCompactionWithHFileLink type FLUSH 2023-03-22 15:55:06,366 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1679500506226 2023-03-22 15:55:06,366 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1679500506226' with timeout 300000ms 2023-03-22 15:55:06,367 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-22 15:55:06,367 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1679500506226' starting 'acquire' stage 2023-03-22 15:55:06,367 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1679500506226' locally acquired 2023-03-22 15:55:06,367 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (snaptb-1679500506226) in zk 2023-03-22 15:55:06,368 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,368 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1679500506226 2023-03-22 15:55:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-22 15:55:06,368 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1679500506226 2023-03-22 15:55:06,368 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1679500506226' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:06,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:06,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:06,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679500506226 2023-03-22 15:55:06,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:06,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'snaptb-1679500506226' on coordinator 2023-03-22 15:55:06,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@639a380f[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:06,370 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1679500506226' starting 'in-barrier' execution. 2023-03-22 15:55:06,370 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1679500506226 2023-03-22 15:55:06,372 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679500506226 2023-03-22 15:55:06,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1679500506226 2023-03-22 15:55:06,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1679500506226 2023-03-22 15:55:06,372 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1679500506226' received 'reached' from coordinator. 2023-03-22 15:55:06,372 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:06,372 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:06,373 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-22 15:55:06,373 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:06,373 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:06,373 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-22 15:55:06,373 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:06,374 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. started... 2023-03-22 15:55:06,374 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. started... 2023-03-22 15:55:06,374 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. started... 2023-03-22 15:55:06,374 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:06,374 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:06,374 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:06,375 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.' region-info for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,375 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.' region-info for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,375 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.' region-info for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,610 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:06,610 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-22 15:55:06,610 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-22 15:55:06,610 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-22 15:55:06,610 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-22 15:55:06,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:06,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/f9075a905dd8452c8f347a85a0de825f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/b8feeb8b3e8c416882ec0c124ebd0f4d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/043c5cd69fe94f4bb70b8fbff9329fee, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/19d38f1e5b9a417fbe004d4965abcf7c] hfiles 2023-03-22 15:55:06,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-22 15:55:06,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/7917639753e84614aefeb4c0d38d5c9d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/03e5d7bf4bbc4f9dba04343703393d6b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/a0ad7a2265a043bc80f148a22e45cb6f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/8a275a5fcb784f13804cf407ba7a368d] hfiles 2023-03-22 15:55:06,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/ce9c7293befc4ac2b83081d67235637f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/f3b4e4f5037842a393873a3f040dc476, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/7876950a8aaf4d5eacc03c89aba69af3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/16f191502d2a4ad2bd45864da26c4836] hfiles 2023-03-22 15:55:06,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/7917639753e84614aefeb4c0d38d5c9d for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,615 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-22 15:55:06,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/f9075a905dd8452c8f347a85a0de825f for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,614 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/ce9c7293befc4ac2b83081d67235637f for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,618 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/f3b4e4f5037842a393873a3f040dc476 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,618 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/b8feeb8b3e8c416882ec0c124ebd0f4d for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/043c5cd69fe94f4bb70b8fbff9329fee for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/19d38f1e5b9a417fbe004d4965abcf7c for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7abb4a6c3d5d43a894a2c727b03c262a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/70e5363498bd448089cef77631a32139, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/c9aaf840692146c29c916b1885331b30, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/18e0114e20734d5c87b318ace88eb8a2] hfiles 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7abb4a6c3d5d43a894a2c727b03c262a for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/70e5363498bd448089cef77631a32139 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/c9aaf840692146c29c916b1885331b30 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/18e0114e20734d5c87b318ace88eb8a2 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,618 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/03e5d7bf4bbc4f9dba04343703393d6b for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/a0ad7a2265a043bc80f148a22e45cb6f for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/8a275a5fcb784f13804cf407ba7a368d for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,618 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/7876950a8aaf4d5eacc03c89aba69af3 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/2aa4237f0ab246f69abbc12410b88bba, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/4dd66f50257149a69fc3e0617e774e1c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/8b37dea4807341f996a5efdd042112a0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/a5af574092a54fc19d8d96cf59f9ec91] hfiles 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/16f191502d2a4ad2bd45864da26c4836 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,619 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/2aa4237f0ab246f69abbc12410b88bba for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,620 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/71885845d5394c4f8c6fa2cf66aee993, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/a17d54ef6c504ff5aa86672257e08710, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/ae4e7e46dd414d1794011c71e8b2d40d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/9f3ebcd7510d453d968b5291f038a60a] hfiles 2023-03-22 15:55:06,620 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/71885845d5394c4f8c6fa2cf66aee993 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,620 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/4dd66f50257149a69fc3e0617e774e1c for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,620 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/a17d54ef6c504ff5aa86672257e08710 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,620 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/8b37dea4807341f996a5efdd042112a0 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,620 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/ae4e7e46dd414d1794011c71e8b2d40d for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,620 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/a5af574092a54fc19d8d96cf59f9ec91 for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,620 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/9f3ebcd7510d453d968b5291f038a60a for snapshot=snaptb-1679500506226 2023-03-22 15:55:06,815 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:06,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:06,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-22 15:55:06,816 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-22 15:55:07,046 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. completed. 2023-03-22 15:55:07,046 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:07,047 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. completed. 2023-03-22 15:55:07,047 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:07,047 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. completed. 2023-03-22 15:55:07,047 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-22 15:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-22 15:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-22 15:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-22 15:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1679500506226' locally completed 2023-03-22 15:55:07,053 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1679500506226' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:07,056 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1679500506226' has notified controller of completion 2023-03-22 15:55:07,056 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,056 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-22 15:55:07,056 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1679500506226' completed. 2023-03-22 15:55:07,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:07,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:07,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679500506226 2023-03-22 15:55:07,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:07,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679500506226 2023-03-22 15:55:07,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1679500506226' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:07,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'snaptb-1679500506226', counting down latch. Waiting for 0 more 2023-03-22 15:55:07,058 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1679500506226' execution completed 2023-03-22 15:55:07,058 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:07,058 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:07,058 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1679500506226 2023-03-22 15:55:07,058 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1679500506226including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-22 15:55:07,060 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679500506226 2023-03-22 15:55:07,060 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679500506226 2023-03-22 15:55:07,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1679500506226 2023-03-22 15:55:07,060 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-22 15:55:07,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1679500506226 2023-03-22 15:55:07,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1679500506226 2023-03-22 15:55:07,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:07,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-22 15:55:07,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-22 15:55:07,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-22 15:55:07,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:07,061 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1679500506226 2023-03-22 15:55:07,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679500506226 2023-03-22 15:55:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679500506226 2023-03-22 15:55:07,062 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679500506226 2023-03-22 15:55:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,065 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,065 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:07,065 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1679500506226 2023-03-22 15:55:07,065 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-22 15:55:07,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1679500506226 because node does not exist (not an error) 2023-03-22 15:55:07,065 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679500506226 2023-03-22 15:55:07,065 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-22 15:55:07,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-22 15:55:07,065 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-22 15:55:07,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-22 15:55:07,065 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1679500506226 2023-03-22 15:55:07,065 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679500506226 2023-03-22 15:55:07,065 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-22 15:55:07,065 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679500506226/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:07,065 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679500506226 2023-03-22 15:55:07,066 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679500506226 2023-03-22 15:55:07,066 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679500506226 2023-03-22 15:55:07,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-22 15:55:07,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032292092a0688b74b8292c4011b93e1193a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322b5802331476e4c4d8d93c74275b69e33, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322c616bcebc940418b8c1df3a672466bc6, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322e5039a9c9c6e462ea98ef70511033233, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303220495f367c8bf4559a0ce2059d3c42779, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322059296d521f74be3822284aae2c125ab, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032262ef361c18a54c10b19099f1a899c95e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303227dae9615fad648e6b55bc956feb5a1e4, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322127ec934bcb24f44b0fdfe9f5c430400, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303227142606f287940e88cf21ec6c6677b0e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303229ff89dda9c844b89bb8dd67fe572459c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322f18742089e924f019e0769cd09b681ed] mob files 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032292092a0688b74b8292c4011b93e1193a 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322b5802331476e4c4d8d93c74275b69e33 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322c616bcebc940418b8c1df3a672466bc6 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322e5039a9c9c6e462ea98ef70511033233 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303220495f367c8bf4559a0ce2059d3c42779 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322059296d521f74be3822284aae2c125ab 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032262ef361c18a54c10b19099f1a899c95e 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303227dae9615fad648e6b55bc956feb5a1e4 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322127ec934bcb24f44b0fdfe9f5c430400 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303227142606f287940e88cf21ec6c6677b0e 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303229ff89dda9c844b89bb8dd67fe572459c 2023-03-22 15:55:07,067 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322f18742089e924f019e0769cd09b681ed 2023-03-22 15:55:07,068 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303222668112f32ab495cb8a4b79b50260eca, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303224e85c149281a40baac2d2780fe54cc0f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032266fce2129b984cbeaf78d4f2e1977b69, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303229a9bc402608441c48b9f5dbc065f00f1, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032208d494d999aa4eaba06d6b4130ed46f2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322320558028cbb41f4a7071f12aabcf0bd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032276687dc503704c4393bed8418c7e7461, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322af819ee46ca44f64b03db21e05ff8841, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303227b9d7305868c40e5a8ffcffeaf6487ad, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322cca9738e1f2b4157bec9d6e75deb0e10, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322d73886fbadb84804a50fc3023579319b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322dd442995df4a42ab96923a9fcf01df24] mob files 2023-03-22 15:55:07,068 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303222668112f32ab495cb8a4b79b50260eca 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303224e85c149281a40baac2d2780fe54cc0f 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032266fce2129b984cbeaf78d4f2e1977b69 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303229a9bc402608441c48b9f5dbc065f00f1 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032208d494d999aa4eaba06d6b4130ed46f2 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322320558028cbb41f4a7071f12aabcf0bd 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032276687dc503704c4393bed8418c7e7461 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322af819ee46ca44f64b03db21e05ff8841 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303227b9d7305868c40e5a8ffcffeaf6487ad 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322cca9738e1f2b4157bec9d6e75deb0e10 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322d73886fbadb84804a50fc3023579319b 2023-03-22 15:55:07,069 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322dd442995df4a42ab96923a9fcf01df24 2023-03-22 15:55:07,116 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:07,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:07,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-22 15:55:07,117 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-22 15:55:07,475 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1679500506226 2023-03-22 15:55:07,477 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/.tmp/snaptb-1679500506226 2023-03-22 15:55:07,617 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:07,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:07,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-22 15:55:07,619 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-22 15:55:07,637 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/.tmp/snaptb-1679500506226 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/snaptb-1679500506226 2023-03-22 15:55:08,092 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1679500506226 of table testCompactionWithHFileLink completed 2023-03-22 15:55:08,093 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/.tmp/snaptb-1679500506226 2023-03-22 15:55:08,093 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/.tmp/snaptb-1679500506226 2023-03-22 15:55:08,093 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1679500506226 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1679500506269Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1679500507475 (+1206 ms)Consolidate snapshot: snaptb-1679500506226 at 1679500507475Loading Region manifests for snaptb-1679500506226 at 1679500507475Writing data manifest for snaptb-1679500506226 at 1679500507535 (+60 ms)Verifying snapshot: snaptb-1679500506226 at 1679500507580 (+45 ms)Snapshot snaptb-1679500506226 of table testCompactionWithHFileLink completed at 1679500508092 (+512 ms) 2023-03-22 15:55:08,095 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-22 15:55:08,097 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.8250 sec 2023-03-22 15:55:08,619 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:08,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:08,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-22 15:55:08,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:08,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:08,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-22 15:55:08,655 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-22 15:55:08,655 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:08,655 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-22 15:55:08,655 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-22 15:55:08,656 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:08,656 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:08,657 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:08,657 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:08,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:08,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:08,658 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:08,658 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:08,658 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:08,658 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-22 15:55:08,658 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-22 15:55:08,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-22 15:55:08,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:08,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-22 15:55:08,659 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:08,659 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. started... 2023-03-22 15:55:08,659 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fc08f2f22b58b2bb5e4b114f03a4e725 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-22 15:55:08,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:08,659 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:08,659 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:08,659 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. started... 2023-03-22 15:55:08,659 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:08,659 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. started... 2023-03-22 15:55:08,659 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b3544ae53f253dc547c7d54a0e776400 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-22 15:55:08,660 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2345ee45559d48c9feb8a4a60984e2c3 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-22 15:55:09,484 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/731081a2534746bda2cab7d1805b6bee 2023-03-22 15:55:09,488 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 731081a2534746bda2cab7d1805b6bee 2023-03-22 15:55:09,496 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/13e0a5ed4d76451692278f4c04388798 2023-03-22 15:55:09,497 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/e4ff16a94d6649f18457e4ef9a77f0f3 2023-03-22 15:55:09,501 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 13e0a5ed4d76451692278f4c04388798 2023-03-22 15:55:09,502 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e4ff16a94d6649f18457e4ef9a77f0f3 2023-03-22 15:55:09,906 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/b89d672043164f4982f832be39a4e9dd 2023-03-22 15:55:09,911 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b89d672043164f4982f832be39a4e9dd 2023-03-22 15:55:09,912 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/731081a2534746bda2cab7d1805b6bee as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/731081a2534746bda2cab7d1805b6bee 2023-03-22 15:55:09,916 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 731081a2534746bda2cab7d1805b6bee 2023-03-22 15:55:09,916 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/731081a2534746bda2cab7d1805b6bee, entries=3, sequenceid=24, filesize=5.0 K 2023-03-22 15:55:09,916 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/b89d672043164f4982f832be39a4e9dd as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/b89d672043164f4982f832be39a4e9dd 2023-03-22 15:55:09,920 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b89d672043164f4982f832be39a4e9dd 2023-03-22 15:55:09,920 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/b89d672043164f4982f832be39a4e9dd, entries=1, sequenceid=24, filesize=5.0 K 2023-03-22 15:55:09,921 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for fc08f2f22b58b2bb5e4b114f03a4e725 in 1262ms, sequenceid=24, compaction requested=true 2023-03-22 15:55:09,921 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:09,921 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:09,922 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:10,330 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/7c61e561a39e495baa36ec5e3bf7053c 2023-03-22 15:55:10,333 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/8e6aabbf35264d15a66d8f4d2ee2bf8f 2023-03-22 15:55:10,334 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7c61e561a39e495baa36ec5e3bf7053c 2023-03-22 15:55:10,335 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/13e0a5ed4d76451692278f4c04388798 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/13e0a5ed4d76451692278f4c04388798 2023-03-22 15:55:10,337 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e6aabbf35264d15a66d8f4d2ee2bf8f 2023-03-22 15:55:10,338 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/e4ff16a94d6649f18457e4ef9a77f0f3 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/e4ff16a94d6649f18457e4ef9a77f0f3 2023-03-22 15:55:10,339 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 13e0a5ed4d76451692278f4c04388798 2023-03-22 15:55:10,339 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/13e0a5ed4d76451692278f4c04388798, entries=3, sequenceid=24, filesize=5.0 K 2023-03-22 15:55:10,340 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/7c61e561a39e495baa36ec5e3bf7053c as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7c61e561a39e495baa36ec5e3bf7053c 2023-03-22 15:55:10,342 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e4ff16a94d6649f18457e4ef9a77f0f3 2023-03-22 15:55:10,342 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/e4ff16a94d6649f18457e4ef9a77f0f3, entries=3, sequenceid=24, filesize=5.0 K 2023-03-22 15:55:10,343 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/8e6aabbf35264d15a66d8f4d2ee2bf8f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/8e6aabbf35264d15a66d8f4d2ee2bf8f 2023-03-22 15:55:10,345 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7c61e561a39e495baa36ec5e3bf7053c 2023-03-22 15:55:10,345 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7c61e561a39e495baa36ec5e3bf7053c, entries=1, sequenceid=24, filesize=5.0 K 2023-03-22 15:55:10,346 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b3544ae53f253dc547c7d54a0e776400 in 1687ms, sequenceid=24, compaction requested=true 2023-03-22 15:55:10,346 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:10,346 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:10,346 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:10,347 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e6aabbf35264d15a66d8f4d2ee2bf8f 2023-03-22 15:55:10,347 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/8e6aabbf35264d15a66d8f4d2ee2bf8f, entries=1, sequenceid=24, filesize=5.0 K 2023-03-22 15:55:10,347 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 2345ee45559d48c9feb8a4a60984e2c3 in 1687ms, sequenceid=24, compaction requested=true 2023-03-22 15:55:10,347 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:10,347 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:10,347 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:10,348 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:10,348 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,348 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-22 15:55:10,348 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-22 15:55:10,350 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,350 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:10,350 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:10,350 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:10,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-22 15:55:10,352 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-22 15:55:10,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75ad3bbf[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:10,352 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:10,356 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:10,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:10,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:10,356 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-22 15:55:10,356 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-22 15:55:10,356 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,356 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:10,356 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:10,357 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,357 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-22 15:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:10,358 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:10,358 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-22 15:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:10,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:10,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:10,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:10,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-22 15:55:10,360 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-22 15:55:10,360 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:10,360 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:10,360 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-22 15:55:10,360 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:10,363 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:10,363 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:10,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:10,363 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:10,364 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:10,365 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-22 15:55:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,367 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,367 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:10,367 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:10,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-22 15:55:10,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:10,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-22 15:55:10,367 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-22 15:55:10,368 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:10,368 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:10,368 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-22 15:55:10,368 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-22 15:55:10,369 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:10,369 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:10,367 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:10,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:10,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:10,367 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:10,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:10,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:10,469 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:10,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:10,470 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:10,471 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): fc08f2f22b58b2bb5e4b114f03a4e725/family1 is initiating major compaction (all files) 2023-03-22 15:55:10,471 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:10,471 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:10,471 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of fc08f2f22b58b2bb5e4b114f03a4e725/family1 in testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:10,471 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/7917639753e84614aefeb4c0d38d5c9d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/03e5d7bf4bbc4f9dba04343703393d6b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/a0ad7a2265a043bc80f148a22e45cb6f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/8a275a5fcb784f13804cf407ba7a368d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/731081a2534746bda2cab7d1805b6bee] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp, totalSize=26.7 K 2023-03-22 15:55:10,472 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 7917639753e84614aefeb4c0d38d5c9d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499419 2023-03-22 15:55:10,472 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 03e5d7bf4bbc4f9dba04343703393d6b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:10,472 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting a0ad7a2265a043bc80f148a22e45cb6f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:10,473 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 8a275a5fcb784f13804cf407ba7a368d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:10,473 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 731081a2534746bda2cab7d1805b6bee, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-22 15:55:10,486 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): fc08f2f22b58b2bb5e4b114f03a4e725#family1#compaction#108 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:10,608 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:55:10,899 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230322157ae51326494b8482ee19e206d06a2a_del 2023-03-22 15:55:10,900 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322157ae51326494b8482ee19e206d06a2a_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322157ae51326494b8482ee19e206d06a2a_del 2023-03-22 15:55:10,913 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fcfc3b2ebe424376aafb86a1a1b3d48f 2023-03-22 15:55:10,914 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/fcfc3b2ebe424376aafb86a1a1b3d48f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/fcfc3b2ebe424376aafb86a1a1b3d48f 2023-03-22 15:55:10,917 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fcfc3b2ebe424376aafb86a1a1b3d48f 2023-03-22 15:55:10,924 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fc08f2f22b58b2bb5e4b114f03a4e725/family1 of fc08f2f22b58b2bb5e4b114f03a4e725 into fcfc3b2ebe424376aafb86a1a1b3d48f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:10,925 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:10,925 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:10,925 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): fc08f2f22b58b2bb5e4b114f03a4e725/family2 is initiating major compaction (all files) 2023-03-22 15:55:10,925 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:10,925 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:10,925 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of fc08f2f22b58b2bb5e4b114f03a4e725/family2 in testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:10,925 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/2aa4237f0ab246f69abbc12410b88bba, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/4dd66f50257149a69fc3e0617e774e1c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/8b37dea4807341f996a5efdd042112a0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/a5af574092a54fc19d8d96cf59f9ec91, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/b89d672043164f4982f832be39a4e9dd] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp, totalSize=25.4 K 2023-03-22 15:55:10,926 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 2aa4237f0ab246f69abbc12410b88bba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499419 2023-03-22 15:55:10,926 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 4dd66f50257149a69fc3e0617e774e1c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:10,927 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 8b37dea4807341f996a5efdd042112a0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:10,927 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting a5af574092a54fc19d8d96cf59f9ec91, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:10,928 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting b89d672043164f4982f832be39a4e9dd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-22 15:55:10,944 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): fc08f2f22b58b2bb5e4b114f03a4e725#family2#compaction#109 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:11,011 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/7917639753e84614aefeb4c0d38d5c9d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/03e5d7bf4bbc4f9dba04343703393d6b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/a0ad7a2265a043bc80f148a22e45cb6f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/8a275a5fcb784f13804cf407ba7a368d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/731081a2534746bda2cab7d1805b6bee] to archive 2023-03-22 15:55:11,012 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:11,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/7917639753e84614aefeb4c0d38d5c9d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/7917639753e84614aefeb4c0d38d5c9d 2023-03-22 15:55:11,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/03e5d7bf4bbc4f9dba04343703393d6b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/03e5d7bf4bbc4f9dba04343703393d6b 2023-03-22 15:55:11,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/a0ad7a2265a043bc80f148a22e45cb6f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/a0ad7a2265a043bc80f148a22e45cb6f 2023-03-22 15:55:11,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/8a275a5fcb784f13804cf407ba7a368d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/8a275a5fcb784f13804cf407ba7a368d 2023-03-22 15:55:11,021 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/731081a2534746bda2cab7d1805b6bee to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/731081a2534746bda2cab7d1805b6bee 2023-03-22 15:55:11,354 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230322159f0df59fe842cea306d10b6dc59c9c_del 2023-03-22 15:55:11,354 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322159f0df59fe842cea306d10b6dc59c9c_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230322159f0df59fe842cea306d10b6dc59c9c_del 2023-03-22 15:55:11,783 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b9c010fde9843fdac09981f04bb38f0 2023-03-22 15:55:11,784 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/7b9c010fde9843fdac09981f04bb38f0 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/7b9c010fde9843fdac09981f04bb38f0 2023-03-22 15:55:11,788 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b9c010fde9843fdac09981f04bb38f0 2023-03-22 15:55:11,789 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fc08f2f22b58b2bb5e4b114f03a4e725/family2 of fc08f2f22b58b2bb5e4b114f03a4e725 into 7b9c010fde9843fdac09981f04bb38f0(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:11,789 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:11,789 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:11,789 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): b3544ae53f253dc547c7d54a0e776400/family1 is initiating major compaction (all files) 2023-03-22 15:55:11,790 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:11,790 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:11,790 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of b3544ae53f253dc547c7d54a0e776400/family1 in testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:11,790 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/f9075a905dd8452c8f347a85a0de825f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/b8feeb8b3e8c416882ec0c124ebd0f4d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/043c5cd69fe94f4bb70b8fbff9329fee, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/19d38f1e5b9a417fbe004d4965abcf7c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/13e0a5ed4d76451692278f4c04388798] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp, totalSize=26.7 K 2023-03-22 15:55:11,790 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting f9075a905dd8452c8f347a85a0de825f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499420 2023-03-22 15:55:11,791 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting b8feeb8b3e8c416882ec0c124ebd0f4d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:11,791 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 043c5cd69fe94f4bb70b8fbff9329fee, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:11,792 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 19d38f1e5b9a417fbe004d4965abcf7c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:11,792 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 13e0a5ed4d76451692278f4c04388798, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-22 15:55:11,804 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): b3544ae53f253dc547c7d54a0e776400#family1#compaction#110 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:11,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/2aa4237f0ab246f69abbc12410b88bba, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/4dd66f50257149a69fc3e0617e774e1c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/8b37dea4807341f996a5efdd042112a0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/a5af574092a54fc19d8d96cf59f9ec91, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/b89d672043164f4982f832be39a4e9dd] to archive 2023-03-22 15:55:11,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:11,814 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/2aa4237f0ab246f69abbc12410b88bba to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/2aa4237f0ab246f69abbc12410b88bba 2023-03-22 15:55:11,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/4dd66f50257149a69fc3e0617e774e1c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/4dd66f50257149a69fc3e0617e774e1c 2023-03-22 15:55:11,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/8b37dea4807341f996a5efdd042112a0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/8b37dea4807341f996a5efdd042112a0 2023-03-22 15:55:11,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/a5af574092a54fc19d8d96cf59f9ec91 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/a5af574092a54fc19d8d96cf59f9ec91 2023-03-22 15:55:11,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/b89d672043164f4982f832be39a4e9dd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/b89d672043164f4982f832be39a4e9dd 2023-03-22 15:55:12,213 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230322376bfc4ab87945eaac6e92da555a07dc_del 2023-03-22 15:55:12,213 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322376bfc4ab87945eaac6e92da555a07dc_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322376bfc4ab87945eaac6e92da555a07dc_del 2023-03-22 15:55:13,028 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ef9c05fbe0149c889f326516711b986 2023-03-22 15:55:13,032 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/1ef9c05fbe0149c889f326516711b986 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/1ef9c05fbe0149c889f326516711b986 2023-03-22 15:55:13,036 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ef9c05fbe0149c889f326516711b986 2023-03-22 15:55:13,037 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b3544ae53f253dc547c7d54a0e776400/family1 of b3544ae53f253dc547c7d54a0e776400 into 1ef9c05fbe0149c889f326516711b986(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:13,037 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:13,037 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:13,037 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): b3544ae53f253dc547c7d54a0e776400/family2 is initiating major compaction (all files) 2023-03-22 15:55:13,037 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:13,037 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:13,037 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of b3544ae53f253dc547c7d54a0e776400/family2 in testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:13,038 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7abb4a6c3d5d43a894a2c727b03c262a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/70e5363498bd448089cef77631a32139, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/c9aaf840692146c29c916b1885331b30, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/18e0114e20734d5c87b318ace88eb8a2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7c61e561a39e495baa36ec5e3bf7053c] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp, totalSize=25.4 K 2023-03-22 15:55:13,038 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 7abb4a6c3d5d43a894a2c727b03c262a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499420 2023-03-22 15:55:13,038 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 70e5363498bd448089cef77631a32139, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:13,039 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting c9aaf840692146c29c916b1885331b30, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:13,039 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 18e0114e20734d5c87b318ace88eb8a2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:13,039 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 7c61e561a39e495baa36ec5e3bf7053c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-22 15:55:13,060 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): b3544ae53f253dc547c7d54a0e776400#family2#compaction#111 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:13,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/f9075a905dd8452c8f347a85a0de825f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/b8feeb8b3e8c416882ec0c124ebd0f4d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/043c5cd69fe94f4bb70b8fbff9329fee, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/19d38f1e5b9a417fbe004d4965abcf7c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/13e0a5ed4d76451692278f4c04388798] to archive 2023-03-22 15:55:13,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:13,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/f9075a905dd8452c8f347a85a0de825f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/f9075a905dd8452c8f347a85a0de825f 2023-03-22 15:55:13,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/b8feeb8b3e8c416882ec0c124ebd0f4d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/b8feeb8b3e8c416882ec0c124ebd0f4d 2023-03-22 15:55:13,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/043c5cd69fe94f4bb70b8fbff9329fee to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/043c5cd69fe94f4bb70b8fbff9329fee 2023-03-22 15:55:13,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/19d38f1e5b9a417fbe004d4965abcf7c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/19d38f1e5b9a417fbe004d4965abcf7c 2023-03-22 15:55:13,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/13e0a5ed4d76451692278f4c04388798 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/13e0a5ed4d76451692278f4c04388798 2023-03-22 15:55:13,468 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023032227d9c3cc027b48668c971b46d46f8e78_del 2023-03-22 15:55:13,469 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032227d9c3cc027b48668c971b46d46f8e78_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032227d9c3cc027b48668c971b46d46f8e78_del 2023-03-22 15:55:13,883 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0bac59461494343932dc63a72a70bfb 2023-03-22 15:55:13,884 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/f0bac59461494343932dc63a72a70bfb as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/f0bac59461494343932dc63a72a70bfb 2023-03-22 15:55:13,888 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0bac59461494343932dc63a72a70bfb 2023-03-22 15:55:13,889 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b3544ae53f253dc547c7d54a0e776400/family2 of b3544ae53f253dc547c7d54a0e776400 into f0bac59461494343932dc63a72a70bfb(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:13,889 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:13,889 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:13,889 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 2345ee45559d48c9feb8a4a60984e2c3/family1 is initiating major compaction (all files) 2023-03-22 15:55:13,889 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:13,889 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:13,889 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 2345ee45559d48c9feb8a4a60984e2c3/family1 in testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:13,889 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/ce9c7293befc4ac2b83081d67235637f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/f3b4e4f5037842a393873a3f040dc476, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/7876950a8aaf4d5eacc03c89aba69af3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/16f191502d2a4ad2bd45864da26c4836, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/e4ff16a94d6649f18457e4ef9a77f0f3] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp, totalSize=26.7 K 2023-03-22 15:55:13,890 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting ce9c7293befc4ac2b83081d67235637f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499417 2023-03-22 15:55:13,890 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting f3b4e4f5037842a393873a3f040dc476, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:13,891 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 7876950a8aaf4d5eacc03c89aba69af3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:13,891 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 16f191502d2a4ad2bd45864da26c4836, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:13,892 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting e4ff16a94d6649f18457e4ef9a77f0f3, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-22 15:55:13,907 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 2345ee45559d48c9feb8a4a60984e2c3#family1#compaction#112 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:13,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7abb4a6c3d5d43a894a2c727b03c262a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/70e5363498bd448089cef77631a32139, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/c9aaf840692146c29c916b1885331b30, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/18e0114e20734d5c87b318ace88eb8a2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7c61e561a39e495baa36ec5e3bf7053c] to archive 2023-03-22 15:55:13,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:13,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7abb4a6c3d5d43a894a2c727b03c262a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7abb4a6c3d5d43a894a2c727b03c262a 2023-03-22 15:55:13,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/70e5363498bd448089cef77631a32139 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/70e5363498bd448089cef77631a32139 2023-03-22 15:55:13,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/c9aaf840692146c29c916b1885331b30 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/c9aaf840692146c29c916b1885331b30 2023-03-22 15:55:13,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/18e0114e20734d5c87b318ace88eb8a2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/18e0114e20734d5c87b318ace88eb8a2 2023-03-22 15:55:13,918 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7c61e561a39e495baa36ec5e3bf7053c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/7c61e561a39e495baa36ec5e3bf7053c 2023-03-22 15:55:14,316 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230322c1d16e7bc8544798ae899a6507eeeee5_del 2023-03-22 15:55:14,316 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322c1d16e7bc8544798ae899a6507eeeee5_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c1d16e7bc8544798ae899a6507eeeee5_del 2023-03-22 15:55:15,131 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25245a1191b74218b4b867972b007b72 2023-03-22 15:55:15,132 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/25245a1191b74218b4b867972b007b72 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/25245a1191b74218b4b867972b007b72 2023-03-22 15:55:15,139 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25245a1191b74218b4b867972b007b72 2023-03-22 15:55:15,140 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2345ee45559d48c9feb8a4a60984e2c3/family1 of 2345ee45559d48c9feb8a4a60984e2c3 into 25245a1191b74218b4b867972b007b72(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:15,141 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:15,141 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:15,141 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 2345ee45559d48c9feb8a4a60984e2c3/family2 is initiating major compaction (all files) 2023-03-22 15:55:15,141 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:15,141 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:15,141 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 2345ee45559d48c9feb8a4a60984e2c3/family2 in testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:15,141 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/71885845d5394c4f8c6fa2cf66aee993, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/a17d54ef6c504ff5aa86672257e08710, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/ae4e7e46dd414d1794011c71e8b2d40d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/9f3ebcd7510d453d968b5291f038a60a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/8e6aabbf35264d15a66d8f4d2ee2bf8f] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp, totalSize=25.4 K 2023-03-22 15:55:15,142 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 71885845d5394c4f8c6fa2cf66aee993, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499417 2023-03-22 15:55:15,143 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting a17d54ef6c504ff5aa86672257e08710, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:15,143 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting ae4e7e46dd414d1794011c71e8b2d40d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:15,143 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 9f3ebcd7510d453d968b5291f038a60a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:15,144 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 8e6aabbf35264d15a66d8f4d2ee2bf8f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-22 15:55:15,158 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 2345ee45559d48c9feb8a4a60984e2c3#family2#compaction#113 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:15,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/ce9c7293befc4ac2b83081d67235637f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/f3b4e4f5037842a393873a3f040dc476, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/7876950a8aaf4d5eacc03c89aba69af3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/16f191502d2a4ad2bd45864da26c4836, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/e4ff16a94d6649f18457e4ef9a77f0f3] to archive 2023-03-22 15:55:15,212 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:15,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/ce9c7293befc4ac2b83081d67235637f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/ce9c7293befc4ac2b83081d67235637f 2023-03-22 15:55:15,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/f3b4e4f5037842a393873a3f040dc476 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/f3b4e4f5037842a393873a3f040dc476 2023-03-22 15:55:15,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/7876950a8aaf4d5eacc03c89aba69af3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/7876950a8aaf4d5eacc03c89aba69af3 2023-03-22 15:55:15,219 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/16f191502d2a4ad2bd45864da26c4836 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/16f191502d2a4ad2bd45864da26c4836 2023-03-22 15:55:15,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/e4ff16a94d6649f18457e4ef9a77f0f3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/e4ff16a94d6649f18457e4ef9a77f0f3 2023-03-22 15:55:15,578 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230322c0ad06c47d3d4dc7adb4bf9b9ee9086b_del 2023-03-22 15:55:15,578 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322c0ad06c47d3d4dc7adb4bf9b9ee9086b_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322c0ad06c47d3d4dc7adb4bf9b9ee9086b_del 2023-03-22 15:55:16,396 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 37ebf507c92c4d17ac53e679a54fc4a7 2023-03-22 15:55:16,397 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/37ebf507c92c4d17ac53e679a54fc4a7 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/37ebf507c92c4d17ac53e679a54fc4a7 2023-03-22 15:55:16,402 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 37ebf507c92c4d17ac53e679a54fc4a7 2023-03-22 15:55:16,403 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2345ee45559d48c9feb8a4a60984e2c3/family2 of 2345ee45559d48c9feb8a4a60984e2c3 into 37ebf507c92c4d17ac53e679a54fc4a7(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:16,403 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:16,410 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:16,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/71885845d5394c4f8c6fa2cf66aee993, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/a17d54ef6c504ff5aa86672257e08710, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/ae4e7e46dd414d1794011c71e8b2d40d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/9f3ebcd7510d453d968b5291f038a60a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/8e6aabbf35264d15a66d8f4d2ee2bf8f] to archive 2023-03-22 15:55:16,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:16,420 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/71885845d5394c4f8c6fa2cf66aee993 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/71885845d5394c4f8c6fa2cf66aee993 2023-03-22 15:55:16,422 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/a17d54ef6c504ff5aa86672257e08710 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/a17d54ef6c504ff5aa86672257e08710 2023-03-22 15:55:16,423 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/ae4e7e46dd414d1794011c71e8b2d40d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/ae4e7e46dd414d1794011c71e8b2d40d 2023-03-22 15:55:16,425 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:16,425 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/9f3ebcd7510d453d968b5291f038a60a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/9f3ebcd7510d453d968b5291f038a60a 2023-03-22 15:55:16,427 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/8e6aabbf35264d15a66d8f4d2ee2bf8f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/8e6aabbf35264d15a66d8f4d2ee2bf8f 2023-03-22 15:55:16,444 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:16,445 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:16,453 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-22 15:55:16,544 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-22 15:55:16,552 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230322c1d16e7bc8544798ae899a6507eeeee5_del 2023-03-22 15:55:16,556 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230322157ae51326494b8482ee19e206d06a2a_del 2023-03-22 15:55:16,565 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230322376bfc4ab87945eaac6e92da555a07dc_del 2023-03-22 15:55:16,565 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:16,566 DEBUG [Listener at localhost/43289] zookeeper.ReadOnlyZKClient(139): Connect 0x121b7dad to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:55:16,595 DEBUG [Listener at localhost/43289] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6df97863, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:55:16,595 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:16,596 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:16,596 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:16,706 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032282588ec20c514ad08e1b0ce7821a738c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032282588ec20c514ad08e1b0ce7821a738c 2023-03-22 15:55:16,706 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:16,720 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:55:16,725 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46006, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:55:16,734 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:55:16,736 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41542, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:55:16,745 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322/family1/11067c663a3940369a58c2d590345b85 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-22 15:55:16,745 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322/family1/11067c663a3940369a58c2d590345b85}] 2023-03-22 15:55:16,751 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322/family1/11067c663a3940369a58c2d590345b85 for inclusion in b3544ae53f253dc547c7d54a0e776400/family1 2023-03-22 15:55:16,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-22 15:55:16,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-22 15:55:16,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:16,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322/family1/11067c663a3940369a58c2d590345b85 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__g2vtg5oo9h9cc3ttdk868o5t1cmiti5jp4uvspet88esu5518726oqcerl5bcv9n/family1/11067c663a3940369a58c2d590345b85 2023-03-22 15:55:16,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__g2vtg5oo9h9cc3ttdk868o5t1cmiti5jp4uvspet88esu5518726oqcerl5bcv9n/family1/11067c663a3940369a58c2d590345b85 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/3b2c280ff09b43ceb3d9d960b585efee_SeqId_29_ 2023-03-22 15:55:16,756 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__g2vtg5oo9h9cc3ttdk868o5t1cmiti5jp4uvspet88esu5518726oqcerl5bcv9n/family1/11067c663a3940369a58c2d590345b85 into b3544ae53f253dc547c7d54a0e776400/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/3b2c280ff09b43ceb3d9d960b585efee_SeqId_29_ - updating store file list. 2023-03-22 15:55:16,759 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/3b2c280ff09b43ceb3d9d960b585efee_SeqId_29_ into b3544ae53f253dc547c7d54a0e776400/family1 2023-03-22 15:55:16,760 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__g2vtg5oo9h9cc3ttdk868o5t1cmiti5jp4uvspet88esu5518726oqcerl5bcv9n/family1/11067c663a3940369a58c2d590345b85 into b3544ae53f253dc547c7d54a0e776400/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/3b2c280ff09b43ceb3d9d960b585efee_SeqId_29_) 2023-03-22 15:55:16,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__g2vtg5oo9h9cc3ttdk868o5t1cmiti5jp4uvspet88esu5518726oqcerl5bcv9n/family1/11067c663a3940369a58c2d590345b85 2023-03-22 15:55:16,762 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:16,762 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:16,763 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303220495f367c8bf4559a0ce2059d3c42779 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303220495f367c8bf4559a0ce2059d3c42779 2023-03-22 15:55:16,764 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322059296d521f74be3822284aae2c125ab to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322059296d521f74be3822284aae2c125ab 2023-03-22 15:55:16,765 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032262ef361c18a54c10b19099f1a899c95e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032262ef361c18a54c10b19099f1a899c95e 2023-03-22 15:55:16,766 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303227dae9615fad648e6b55bc956feb5a1e4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303227dae9615fad648e6b55bc956feb5a1e4 2023-03-22 15:55:16,766 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-22 15:55:16,766 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322' 2023-03-22 15:55:16,766 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:17,099 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322071ddd820abe493ba20518009c56b595 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322071ddd820abe493ba20518009c56b595 2023-03-22 15:55:17,100 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:17,133 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322/family1/22bd9392412046d6ae3548b1cfb62d30 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-22 15:55:17,134 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322/family1/22bd9392412046d6ae3548b1cfb62d30}] 2023-03-22 15:55:17,136 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322/family1/22bd9392412046d6ae3548b1cfb62d30 for inclusion in fc08f2f22b58b2bb5e4b114f03a4e725/family1 2023-03-22 15:55:17,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-22 15:55:17,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-22 15:55:17,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:17,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322/family1/22bd9392412046d6ae3548b1cfb62d30 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__4pa4nu95jiserlq36kuo0iiln564v93icvjitoqfd8dljbt5b8dv6fkptlgdnvq9/family1/22bd9392412046d6ae3548b1cfb62d30 2023-03-22 15:55:17,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__4pa4nu95jiserlq36kuo0iiln564v93icvjitoqfd8dljbt5b8dv6fkptlgdnvq9/family1/22bd9392412046d6ae3548b1cfb62d30 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3d3540730d3e46528cb9126ff8a1f5d1_SeqId_29_ 2023-03-22 15:55:17,142 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__4pa4nu95jiserlq36kuo0iiln564v93icvjitoqfd8dljbt5b8dv6fkptlgdnvq9/family1/22bd9392412046d6ae3548b1cfb62d30 into fc08f2f22b58b2bb5e4b114f03a4e725/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3d3540730d3e46528cb9126ff8a1f5d1_SeqId_29_ - updating store file list. 2023-03-22 15:55:17,145 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3d3540730d3e46528cb9126ff8a1f5d1_SeqId_29_ into fc08f2f22b58b2bb5e4b114f03a4e725/family1 2023-03-22 15:55:17,145 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__4pa4nu95jiserlq36kuo0iiln564v93icvjitoqfd8dljbt5b8dv6fkptlgdnvq9/family1/22bd9392412046d6ae3548b1cfb62d30 into fc08f2f22b58b2bb5e4b114f03a4e725/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3d3540730d3e46528cb9126ff8a1f5d1_SeqId_29_) 2023-03-22 15:55:17,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__4pa4nu95jiserlq36kuo0iiln564v93icvjitoqfd8dljbt5b8dv6fkptlgdnvq9/family1/22bd9392412046d6ae3548b1cfb62d30 2023-03-22 15:55:17,147 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:17,148 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:17,149 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032292092a0688b74b8292c4011b93e1193a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032292092a0688b74b8292c4011b93e1193a 2023-03-22 15:55:17,150 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322b5802331476e4c4d8d93c74275b69e33 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322b5802331476e4c4d8d93c74275b69e33 2023-03-22 15:55:17,151 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322c616bcebc940418b8c1df3a672466bc6 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322c616bcebc940418b8c1df3a672466bc6 2023-03-22 15:55:17,151 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322e5039a9c9c6e462ea98ef70511033233 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322e5039a9c9c6e462ea98ef70511033233 2023-03-22 15:55:17,152 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-22 15:55:17,152 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322' 2023-03-22 15:55:17,152 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:17,505 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322c99c0b2c1b8049cfad131d9929957fd7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c99c0b2c1b8049cfad131d9929957fd7 2023-03-22 15:55:17,506 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:17,537 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322/family1/38da7c540d1c41ca9e8cee29240ee9f5 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-22 15:55:17,538 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322/family1/38da7c540d1c41ca9e8cee29240ee9f5}] 2023-03-22 15:55:17,540 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322/family1/38da7c540d1c41ca9e8cee29240ee9f5 for inclusion in 2345ee45559d48c9feb8a4a60984e2c3/family1 2023-03-22 15:55:17,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-22 15:55:17,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last=1 2023-03-22 15:55:17,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:17,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322/family1/38da7c540d1c41ca9e8cee29240ee9f5 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__38er3oet0cquldm5mm82ohpv71qhvc5k72v88dob6ee2lhptoog85dtopesv0ooi/family1/38da7c540d1c41ca9e8cee29240ee9f5 2023-03-22 15:55:17,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__38er3oet0cquldm5mm82ohpv71qhvc5k72v88dob6ee2lhptoog85dtopesv0ooi/family1/38da7c540d1c41ca9e8cee29240ee9f5 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/264f049da9554096820ecbc057157dcb_SeqId_29_ 2023-03-22 15:55:17,547 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__38er3oet0cquldm5mm82ohpv71qhvc5k72v88dob6ee2lhptoog85dtopesv0ooi/family1/38da7c540d1c41ca9e8cee29240ee9f5 into 2345ee45559d48c9feb8a4a60984e2c3/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/264f049da9554096820ecbc057157dcb_SeqId_29_ - updating store file list. 2023-03-22 15:55:17,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/264f049da9554096820ecbc057157dcb_SeqId_29_ into 2345ee45559d48c9feb8a4a60984e2c3/family1 2023-03-22 15:55:17,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__38er3oet0cquldm5mm82ohpv71qhvc5k72v88dob6ee2lhptoog85dtopesv0ooi/family1/38da7c540d1c41ca9e8cee29240ee9f5 into 2345ee45559d48c9feb8a4a60984e2c3/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/264f049da9554096820ecbc057157dcb_SeqId_29_) 2023-03-22 15:55:17,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__38er3oet0cquldm5mm82ohpv71qhvc5k72v88dob6ee2lhptoog85dtopesv0ooi/family1/38da7c540d1c41ca9e8cee29240ee9f5 2023-03-22 15:55:17,560 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:17,561 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:17,564 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322127ec934bcb24f44b0fdfe9f5c430400 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322127ec934bcb24f44b0fdfe9f5c430400 2023-03-22 15:55:17,566 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303227142606f287940e88cf21ec6c6677b0e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303227142606f287940e88cf21ec6c6677b0e 2023-03-22 15:55:17,567 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303229ff89dda9c844b89bb8dd67fe572459c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303229ff89dda9c844b89bb8dd67fe572459c 2023-03-22 15:55:17,569 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322f18742089e924f019e0769cd09b681ed to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322f18742089e924f019e0769cd09b681ed 2023-03-22 15:55:17,569 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-22 15:55:17,569 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322' 2023-03-22 15:55:17,569 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:17,569 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:17,569 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:17,570 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c1d16e7bc8544798ae899a6507eeeee5_del] 2023-03-22 15:55:17,570 DEBUG [Listener at localhost/43289] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:17,572 DEBUG [Listener at localhost/43289] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c1d16e7bc8544798ae899a6507eeeee5_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c1d16e7bc8544798ae899a6507eeeee5_del 2023-03-22 15:55:17,572 INFO [Listener at localhost/43289] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-22 15:55:17,572 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322157ae51326494b8482ee19e206d06a2a_del] 2023-03-22 15:55:17,572 DEBUG [Listener at localhost/43289] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:17,574 DEBUG [Listener at localhost/43289] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322157ae51326494b8482ee19e206d06a2a_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322157ae51326494b8482ee19e206d06a2a_del 2023-03-22 15:55:17,574 INFO [Listener at localhost/43289] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-22 15:55:17,574 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322376bfc4ab87945eaac6e92da555a07dc_del] 2023-03-22 15:55:17,574 DEBUG [Listener at localhost/43289] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:17,575 DEBUG [Listener at localhost/43289] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322376bfc4ab87945eaac6e92da555a07dc_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322376bfc4ab87945eaac6e92da555a07dc_del 2023-03-22 15:55:17,575 INFO [Listener at localhost/43289] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-22 15:55:17,578 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:17,580 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:17,583 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:17,584 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:17,590 INFO [Listener at localhost/43289] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-22 15:55:17,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-22 15:55:17,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-22 15:55:17,594 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500517594"}]},"ts":"1679500517594"} 2023-03-22 15:55:17,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-22 15:55:17,595 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-22 15:55:17,597 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-22 15:55:17,598 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, UNASSIGN}] 2023-03-22 15:55:17,599 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, UNASSIGN 2023-03-22 15:55:17,599 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, UNASSIGN 2023-03-22 15:55:17,600 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, UNASSIGN 2023-03-22 15:55:17,600 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=fc08f2f22b58b2bb5e4b114f03a4e725, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:17,600 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=b3544ae53f253dc547c7d54a0e776400, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:17,600 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=2345ee45559d48c9feb8a4a60984e2c3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:17,600 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500517600"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500517600"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500517600"}]},"ts":"1679500517600"} 2023-03-22 15:55:17,601 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500517600"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500517600"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500517600"}]},"ts":"1679500517600"} 2023-03-22 15:55:17,600 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500517600"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500517600"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500517600"}]},"ts":"1679500517600"} 2023-03-22 15:55:17,602 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure fc08f2f22b58b2bb5e4b114f03a4e725, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:17,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure 2345ee45559d48c9feb8a4a60984e2c3, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:17,603 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure b3544ae53f253dc547c7d54a0e776400, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:17,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-22 15:55:17,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:17,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b3544ae53f253dc547c7d54a0e776400, disabling compactions & flushes 2023-03-22 15:55:17,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:17,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:17,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. after waiting 0 ms 2023-03-22 15:55:17,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:17,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-22 15:55:17,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:17,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:17,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:17,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:17,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2345ee45559d48c9feb8a4a60984e2c3, disabling compactions & flushes 2023-03-22 15:55:17,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:17,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:17,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. after waiting 0 ms 2023-03-22 15:55:17,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:17,761 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=b3544ae53f253dc547c7d54a0e776400, regionState=CLOSED 2023-03-22 15:55:17,761 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500517761"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500517761"}]},"ts":"1679500517761"} 2023-03-22 15:55:17,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-03-22 15:55:17,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure b3544ae53f253dc547c7d54a0e776400, server=jenkins-hbase4.apache.org,38637,1679500416185 in 159 msec 2023-03-22 15:55:17,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-22 15:55:17,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, UNASSIGN in 166 msec 2023-03-22 15:55:17,766 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:17,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:17,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:17,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:17,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fc08f2f22b58b2bb5e4b114f03a4e725, disabling compactions & flushes 2023-03-22 15:55:17,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:17,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:17,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. after waiting 0 ms 2023-03-22 15:55:17,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:17,768 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=2345ee45559d48c9feb8a4a60984e2c3, regionState=CLOSED 2023-03-22 15:55:17,768 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500517768"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500517768"}]},"ts":"1679500517768"} 2023-03-22 15:55:17,771 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=89 2023-03-22 15:55:17,772 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure 2345ee45559d48c9feb8a4a60984e2c3, server=jenkins-hbase4.apache.org,38637,1679500416185 in 168 msec 2023-03-22 15:55:17,773 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, UNASSIGN in 173 msec 2023-03-22 15:55:17,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-22 15:55:17,774 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:17,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:17,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:17,776 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=fc08f2f22b58b2bb5e4b114f03a4e725, regionState=CLOSED 2023-03-22 15:55:17,776 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500517776"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500517776"}]},"ts":"1679500517776"} 2023-03-22 15:55:17,778 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-03-22 15:55:17,778 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure fc08f2f22b58b2bb5e4b114f03a4e725, server=jenkins-hbase4.apache.org,38637,1679500416185 in 175 msec 2023-03-22 15:55:17,780 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-03-22 15:55:17,780 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, UNASSIGN in 180 msec 2023-03-22 15:55:17,780 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500517780"}]},"ts":"1679500517780"} 2023-03-22 15:55:17,781 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-22 15:55:17,783 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-22 15:55:17,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 193 msec 2023-03-22 15:55:17,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-22 15:55:17,898 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-22 15:55:17,911 INFO [Listener at localhost/43289] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1679500506226-1679500517911 2023-03-22 15:55:17,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-22 15:55:17,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1679500517912 (current time:1679500517912). 2023-03-22 15:55:17,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-22 15:55:17,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] zookeeper.ReadOnlyZKClient(139): Connect 0x4fcbe2f2 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:55:17,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@12740cb9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:55:17,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:55:17,923 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46012, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:55:17,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4fcbe2f2 to 127.0.0.1:58225 2023-03-22 15:55:17,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:17,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-22 15:55:17,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-22 15:55:17,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-22 15:55:17,929 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-22 15:55:17,930 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1679501117930 2023-03-22 15:55:17,930 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1679500506226-1679500517911 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-22 15:55:17,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-22 15:55:17,930 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:17,931 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-22 15:55:18,031 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:18,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:18,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-22 15:55:18,032 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-22 15:55:18,232 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:18,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:18,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-22 15:55:18,233 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-22 15:55:18,343 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-22 15:55:18,345 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-22 15:55:18,345 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-22 15:55:18,348 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-22 15:55:18,348 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-22 15:55:18,348 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-22 15:55:18,348 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-22 15:55:18,349 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-22 15:55:18,349 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/1ef9c05fbe0149c889f326516711b986, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/3b2c280ff09b43ceb3d9d960b585efee_SeqId_29_] hfiles 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3d3540730d3e46528cb9126ff8a1f5d1_SeqId_29_, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/fcfc3b2ebe424376aafb86a1a1b3d48f] hfiles 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/1ef9c05fbe0149c889f326516711b986 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322071ddd820abe493ba20518009c56b595, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032282588ec20c514ad08e1b0ce7821a738c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c99c0b2c1b8049cfad131d9929957fd7] hfiles 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3d3540730d3e46528cb9126ff8a1f5d1_SeqId_29_ 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322071ddd820abe493ba20518009c56b595 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/3b2c280ff09b43ceb3d9d960b585efee_SeqId_29_ 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032282588ec20c514ad08e1b0ce7821a738c 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c99c0b2c1b8049cfad131d9929957fd7 2023-03-22 15:55:18,351 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/fcfc3b2ebe424376aafb86a1a1b3d48f 2023-03-22 15:55:18,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/25245a1191b74218b4b867972b007b72, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/264f049da9554096820ecbc057157dcb_SeqId_29_] hfiles 2023-03-22 15:55:18,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/25245a1191b74218b4b867972b007b72 2023-03-22 15:55:18,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/264f049da9554096820ecbc057157dcb_SeqId_29_ 2023-03-22 15:55:18,353 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/f0bac59461494343932dc63a72a70bfb] hfiles 2023-03-22 15:55:18,353 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/f0bac59461494343932dc63a72a70bfb 2023-03-22 15:55:18,353 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/7b9c010fde9843fdac09981f04bb38f0] hfiles 2023-03-22 15:55:18,353 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/7b9c010fde9843fdac09981f04bb38f0 2023-03-22 15:55:18,353 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/37ebf507c92c4d17ac53e679a54fc4a7] hfiles 2023-03-22 15:55:18,353 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/37ebf507c92c4d17ac53e679a54fc4a7 2023-03-22 15:55:18,363 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230322159f0df59fe842cea306d10b6dc59c9c_del, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303222668112f32ab495cb8a4b79b50260eca, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303224e85c149281a40baac2d2780fe54cc0f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032266fce2129b984cbeaf78d4f2e1977b69, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303229a9bc402608441c48b9f5dbc065f00f1, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032208d494d999aa4eaba06d6b4130ed46f2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032227d9c3cc027b48668c971b46d46f8e78_del, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322320558028cbb41f4a7071f12aabcf0bd, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032276687dc503704c4393bed8418c7e7461, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322af819ee46ca44f64b03db21e05ff8841, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303227b9d7305868c40e5a8ffcffeaf6487ad, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322c0ad06c47d3d4dc7adb4bf9b9ee9086b_del, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322cca9738e1f2b4157bec9d6e75deb0e10, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322d73886fbadb84804a50fc3023579319b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322dd442995df4a42ab96923a9fcf01df24] hfiles 2023-03-22 15:55:18,363 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230322159f0df59fe842cea306d10b6dc59c9c_del 2023-03-22 15:55:18,363 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303222668112f32ab495cb8a4b79b50260eca 2023-03-22 15:55:18,363 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303224e85c149281a40baac2d2780fe54cc0f 2023-03-22 15:55:18,364 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032266fce2129b984cbeaf78d4f2e1977b69 2023-03-22 15:55:18,364 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303229a9bc402608441c48b9f5dbc065f00f1 2023-03-22 15:55:18,364 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032208d494d999aa4eaba06d6b4130ed46f2 2023-03-22 15:55:18,364 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032227d9c3cc027b48668c971b46d46f8e78_del 2023-03-22 15:55:18,364 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322320558028cbb41f4a7071f12aabcf0bd 2023-03-22 15:55:18,364 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032276687dc503704c4393bed8418c7e7461 2023-03-22 15:55:18,364 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230322af819ee46ca44f64b03db21e05ff8841 2023-03-22 15:55:18,364 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303227b9d7305868c40e5a8ffcffeaf6487ad 2023-03-22 15:55:18,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322c0ad06c47d3d4dc7adb4bf9b9ee9086b_del 2023-03-22 15:55:18,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322cca9738e1f2b4157bec9d6e75deb0e10 2023-03-22 15:55:18,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322d73886fbadb84804a50fc3023579319b 2023-03-22 15:55:18,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322dd442995df4a42ab96923a9fcf01df24 2023-03-22 15:55:18,534 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:18,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:18,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-22 15:55:18,535 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-22 15:55:18,770 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-22 15:55:18,770 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1679500506226-1679500517911 2023-03-22 15:55:18,770 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679500506226-1679500517911 2023-03-22 15:55:18,807 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679500506226-1679500517911 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/hbase-failsafe-snaptb-1679500506226-1679500517911 2023-03-22 15:55:19,035 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:19,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:19,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-22 15:55:19,036 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-22 15:55:19,255 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1679500506226-1679500517911 of table testCompactionWithHFileLink completed 2023-03-22 15:55:19,255 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679500506226-1679500517911 2023-03-22 15:55:19,257 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679500506226-1679500517911 2023-03-22 15:55:19,257 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1679500506226-1679500517911 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1679500517930Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1679500518343 (+413 ms)Consolidate snapshot: hbase-failsafe-snaptb-1679500506226-1679500517911 at 1679500518770 (+427 ms)Loading Region manifests for hbase-failsafe-snaptb-1679500506226-1679500517911 at 1679500518770Writing data manifest for hbase-failsafe-snaptb-1679500506226-1679500517911 at 1679500518774 (+4 ms)Verifying snapshot: hbase-failsafe-snaptb-1679500506226-1679500517911 at 1679500518784 (+10 ms)Snapshot hbase-failsafe-snaptb-1679500506226-1679500517911 of table testCompactionWithHFileLink completed at 1679500519255 (+471 ms) 2023-03-22 15:55:19,259 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-22 15:55:19,260 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.3310 sec 2023-03-22 15:55:20,036 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-22 15:55:20,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-22 15:55:20,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1679500506226-1679500517911 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-22 15:55:20,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1679500506226" table: "testCompactionWithHFileLink" creation_time: 1679500506240 type: FLUSH version: 2 ttl: 0 ) 2023-03-22 15:55:20,096 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1679500506226 as table=testCompactionWithHFileLink 2023-03-22 15:55:20,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-22 15:55:20,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-22 15:55:20,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-22 15:55:20,510 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-22 15:55:20,511 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-22 15:55:20,511 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-22 15:55:20,512 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-22 15:55:20,517 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1679500506226" table: "testCompactionWithHFileLink" creation_time: 1679500506240 type: FLUSH version: 2 ttl: 0 2023-03-22 15:55:20,517 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink 2023-03-22 15:55:20,523 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-22 15:55:20,527 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322071ddd820abe493ba20518009c56b595 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322071ddd820abe493ba20518009c56b595 2023-03-22 15:55:20,528 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032282588ec20c514ad08e1b0ce7821a738c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032282588ec20c514ad08e1b0ce7821a738c 2023-03-22 15:55:20,530 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c99c0b2c1b8049cfad131d9929957fd7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c99c0b2c1b8049cfad131d9929957fd7 2023-03-22 15:55:20,530 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023032292092a0688b74b8292c4011b93e1193a of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,533 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230322b5802331476e4c4d8d93c74275b69e33 of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,537 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230322c616bcebc940418b8c1df3a672466bc6 of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,540 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230322e5039a9c9c6e462ea98ef70511033233 of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,543 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303220495f367c8bf4559a0ce2059d3c42779 of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,546 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230322059296d521f74be3822284aae2c125ab of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,549 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023032262ef361c18a54c10b19099f1a899c95e of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,553 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303227dae9615fad648e6b55bc956feb5a1e4 of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,556 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230322127ec934bcb24f44b0fdfe9f5c430400 of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,565 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303227142606f287940e88cf21ec6c6677b0e of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,575 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303229ff89dda9c844b89bb8dd67fe572459c of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,587 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230322f18742089e924f019e0769cd09b681ed of snapshot=snaptb-1679500506226 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-22 15:55:20,595 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230322159f0df59fe842cea306d10b6dc59c9c_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230322159f0df59fe842cea306d10b6dc59c9c_del 2023-03-22 15:55:20,597 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322c0ad06c47d3d4dc7adb4bf9b9ee9086b_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230322c0ad06c47d3d4dc7adb4bf9b9ee9086b_del 2023-03-22 15:55:20,598 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032227d9c3cc027b48668c971b46d46f8e78_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032227d9c3cc027b48668c971b46d46f8e78_del 2023-03-22 15:55:20,598 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:20,599 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:20,599 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:20,607 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/fcfc3b2ebe424376aafb86a1a1b3d48f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/fcfc3b2ebe424376aafb86a1a1b3d48f 2023-03-22 15:55:20,607 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/3b2c280ff09b43ceb3d9d960b585efee_SeqId_29_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/3b2c280ff09b43ceb3d9d960b585efee_SeqId_29_ 2023-03-22 15:55:20,608 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/264f049da9554096820ecbc057157dcb_SeqId_29_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/264f049da9554096820ecbc057157dcb_SeqId_29_ 2023-03-22 15:55:20,610 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3d3540730d3e46528cb9126ff8a1f5d1_SeqId_29_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3d3540730d3e46528cb9126ff8a1f5d1_SeqId_29_ 2023-03-22 15:55:20,610 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7917639753e84614aefeb4c0d38d5c9d of snapshot=snaptb-1679500506226 to region=fc08f2f22b58b2bb5e4b114f03a4e725 table=testCompactionWithHFileLink 2023-03-22 15:55:20,611 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/25245a1191b74218b4b867972b007b72 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/25245a1191b74218b4b867972b007b72 2023-03-22 15:55:20,611 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ce9c7293befc4ac2b83081d67235637f of snapshot=snaptb-1679500506226 to region=2345ee45559d48c9feb8a4a60984e2c3 table=testCompactionWithHFileLink 2023-03-22 15:55:20,616 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/1ef9c05fbe0149c889f326516711b986 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/1ef9c05fbe0149c889f326516711b986 2023-03-22 15:55:20,617 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f9075a905dd8452c8f347a85a0de825f of snapshot=snaptb-1679500506226 to region=b3544ae53f253dc547c7d54a0e776400 table=testCompactionWithHFileLink 2023-03-22 15:55:20,627 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 03e5d7bf4bbc4f9dba04343703393d6b of snapshot=snaptb-1679500506226 to region=fc08f2f22b58b2bb5e4b114f03a4e725 table=testCompactionWithHFileLink 2023-03-22 15:55:20,627 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f3b4e4f5037842a393873a3f040dc476 of snapshot=snaptb-1679500506226 to region=2345ee45559d48c9feb8a4a60984e2c3 table=testCompactionWithHFileLink 2023-03-22 15:55:20,629 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b8feeb8b3e8c416882ec0c124ebd0f4d of snapshot=snaptb-1679500506226 to region=b3544ae53f253dc547c7d54a0e776400 table=testCompactionWithHFileLink 2023-03-22 15:55:20,641 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7876950a8aaf4d5eacc03c89aba69af3 of snapshot=snaptb-1679500506226 to region=2345ee45559d48c9feb8a4a60984e2c3 table=testCompactionWithHFileLink 2023-03-22 15:55:20,643 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a0ad7a2265a043bc80f148a22e45cb6f of snapshot=snaptb-1679500506226 to region=fc08f2f22b58b2bb5e4b114f03a4e725 table=testCompactionWithHFileLink 2023-03-22 15:55:20,643 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 043c5cd69fe94f4bb70b8fbff9329fee of snapshot=snaptb-1679500506226 to region=b3544ae53f253dc547c7d54a0e776400 table=testCompactionWithHFileLink 2023-03-22 15:55:20,660 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 16f191502d2a4ad2bd45864da26c4836 of snapshot=snaptb-1679500506226 to region=2345ee45559d48c9feb8a4a60984e2c3 table=testCompactionWithHFileLink 2023-03-22 15:55:20,660 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 19d38f1e5b9a417fbe004d4965abcf7c of snapshot=snaptb-1679500506226 to region=b3544ae53f253dc547c7d54a0e776400 table=testCompactionWithHFileLink 2023-03-22 15:55:20,661 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8a275a5fcb784f13804cf407ba7a368d of snapshot=snaptb-1679500506226 to region=fc08f2f22b58b2bb5e4b114f03a4e725 table=testCompactionWithHFileLink 2023-03-22 15:55:20,674 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/37ebf507c92c4d17ac53e679a54fc4a7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/37ebf507c92c4d17ac53e679a54fc4a7 2023-03-22 15:55:20,674 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 71885845d5394c4f8c6fa2cf66aee993 of snapshot=snaptb-1679500506226 to region=2345ee45559d48c9feb8a4a60984e2c3 table=testCompactionWithHFileLink 2023-03-22 15:55:20,675 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/f0bac59461494343932dc63a72a70bfb to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/f0bac59461494343932dc63a72a70bfb 2023-03-22 15:55:20,675 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7abb4a6c3d5d43a894a2c727b03c262a of snapshot=snaptb-1679500506226 to region=b3544ae53f253dc547c7d54a0e776400 table=testCompactionWithHFileLink 2023-03-22 15:55:20,682 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/7b9c010fde9843fdac09981f04bb38f0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/7b9c010fde9843fdac09981f04bb38f0 2023-03-22 15:55:20,682 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2aa4237f0ab246f69abbc12410b88bba of snapshot=snaptb-1679500506226 to region=fc08f2f22b58b2bb5e4b114f03a4e725 table=testCompactionWithHFileLink 2023-03-22 15:55:20,695 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4dd66f50257149a69fc3e0617e774e1c of snapshot=snaptb-1679500506226 to region=fc08f2f22b58b2bb5e4b114f03a4e725 table=testCompactionWithHFileLink 2023-03-22 15:55:20,695 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a17d54ef6c504ff5aa86672257e08710 of snapshot=snaptb-1679500506226 to region=2345ee45559d48c9feb8a4a60984e2c3 table=testCompactionWithHFileLink 2023-03-22 15:55:20,695 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 70e5363498bd448089cef77631a32139 of snapshot=snaptb-1679500506226 to region=b3544ae53f253dc547c7d54a0e776400 table=testCompactionWithHFileLink 2023-03-22 15:55:20,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-22 15:55:20,720 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ae4e7e46dd414d1794011c71e8b2d40d of snapshot=snaptb-1679500506226 to region=2345ee45559d48c9feb8a4a60984e2c3 table=testCompactionWithHFileLink 2023-03-22 15:55:20,720 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c9aaf840692146c29c916b1885331b30 of snapshot=snaptb-1679500506226 to region=b3544ae53f253dc547c7d54a0e776400 table=testCompactionWithHFileLink 2023-03-22 15:55:20,720 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8b37dea4807341f996a5efdd042112a0 of snapshot=snaptb-1679500506226 to region=fc08f2f22b58b2bb5e4b114f03a4e725 table=testCompactionWithHFileLink 2023-03-22 15:55:20,730 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9f3ebcd7510d453d968b5291f038a60a of snapshot=snaptb-1679500506226 to region=2345ee45559d48c9feb8a4a60984e2c3 table=testCompactionWithHFileLink 2023-03-22 15:55:20,731 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a5af574092a54fc19d8d96cf59f9ec91 of snapshot=snaptb-1679500506226 to region=fc08f2f22b58b2bb5e4b114f03a4e725 table=testCompactionWithHFileLink 2023-03-22 15:55:20,740 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 18e0114e20734d5c87b318ace88eb8a2 of snapshot=snaptb-1679500506226 to region=b3544ae53f253dc547c7d54a0e776400 table=testCompactionWithHFileLink 2023-03-22 15:55:20,744 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1679500506226" table: "testCompactionWithHFileLink" creation_time: 1679500506240 type: FLUSH version: 2 ttl: 0 2023-03-22 15:55:20,745 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679500520745"}]},"ts":"9223372036854775807"} 2023-03-22 15:55:20,745 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679500520745"}]},"ts":"9223372036854775807"} 2023-03-22 15:55:20,745 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679500520745"}]},"ts":"9223372036854775807"} 2023-03-22 15:55:20,747 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-22 15:55:20,747 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2345ee45559d48c9feb8a4a60984e2c3, NAME => 'testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => b3544ae53f253dc547c7d54a0e776400, NAME => 'testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => fc08f2f22b58b2bb5e4b114f03a4e725, NAME => 'testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-22 15:55:20,748 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500520746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500520746"}]},"ts":"1679500520746"} 2023-03-22 15:55:20,748 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500520746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500520746"}]},"ts":"1679500520746"} 2023-03-22 15:55:20,748 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500520746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500520746"}]},"ts":"1679500520746"} 2023-03-22 15:55:20,750 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-22 15:55:20,750 INFO [PEWorker-2] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-22 15:55:20,750 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 2345ee45559d48c9feb8a4a60984e2c3, NAME => 'testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => b3544ae53f253dc547c7d54a0e776400, NAME => 'testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => fc08f2f22b58b2bb5e4b114f03a4e725, NAME => 'testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-22 15:55:20,751 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1679500506226 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-22 15:55:20,753 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1679500506226" table: "testCompactionWithHFileLink" creation_time: 1679500506240 type: FLUSH version: 2 ttl: 0 ) in 668 msec 2023-03-22 15:55:21,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-22 15:55:21,205 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-22 15:55:21,205 INFO [Listener at localhost/43289] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1679500506226-1679500517911 2023-03-22 15:55:21,210 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1679500506226-1679500517911" 2023-03-22 15:55:21,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1679500506226-1679500517911 2023-03-22 15:55:21,215 INFO [Listener at localhost/43289] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-22 15:55:21,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-22 15:55:21,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-22 15:55:21,230 INFO [PEWorker-5] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-22 15:55:21,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-22 15:55:21,230 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500521230"}]},"ts":"1679500521230"} 2023-03-22 15:55:21,231 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-22 15:55:21,234 DEBUG [PEWorker-5] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-22 15:55:21,234 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, ASSIGN}] 2023-03-22 15:55:21,236 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, ASSIGN 2023-03-22 15:55:21,237 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, ASSIGN 2023-03-22 15:55:21,237 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, ASSIGN 2023-03-22 15:55:21,237 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-22 15:55:21,237 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-22 15:55:21,237 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-22 15:55:21,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-22 15:55:21,389 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=2345ee45559d48c9feb8a4a60984e2c3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:21,389 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=fc08f2f22b58b2bb5e4b114f03a4e725, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:21,389 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=b3544ae53f253dc547c7d54a0e776400, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:21,389 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500521389"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500521389"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500521389"}]},"ts":"1679500521389"} 2023-03-22 15:55:21,389 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500521389"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500521389"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500521389"}]},"ts":"1679500521389"} 2023-03-22 15:55:21,389 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500521389"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500521389"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500521389"}]},"ts":"1679500521389"} 2023-03-22 15:55:21,391 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure b3544ae53f253dc547c7d54a0e776400, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:21,392 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=99, state=RUNNABLE; OpenRegionProcedure fc08f2f22b58b2bb5e4b114f03a4e725, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:21,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure 2345ee45559d48c9feb8a4a60984e2c3, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:21,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-22 15:55:21,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:21,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b3544ae53f253dc547c7d54a0e776400, NAME => 'testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.', STARTKEY => '2', ENDKEY => ''} 2023-03-22 15:55:21,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:21,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:21,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:21,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:21,548 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:21,549 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1 2023-03-22 15:55:21,549 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1 2023-03-22 15:55:21,549 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3544ae53f253dc547c7d54a0e776400 columnFamilyName family1 2023-03-22 15:55:21,568 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-043c5cd69fe94f4bb70b8fbff9329fee 2023-03-22 15:55:21,574 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-19d38f1e5b9a417fbe004d4965abcf7c 2023-03-22 15:55:21,580 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-b8feeb8b3e8c416882ec0c124ebd0f4d 2023-03-22 15:55:21,587 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-f9075a905dd8452c8f347a85a0de825f 2023-03-22 15:55:21,587 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(310): Store=b3544ae53f253dc547c7d54a0e776400/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:21,588 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:21,589 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2 2023-03-22 15:55:21,589 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2 2023-03-22 15:55:21,589 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3544ae53f253dc547c7d54a0e776400 columnFamilyName family2 2023-03-22 15:55:21,599 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-18e0114e20734d5c87b318ace88eb8a2 2023-03-22 15:55:21,609 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-70e5363498bd448089cef77631a32139 2023-03-22 15:55:21,615 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-7abb4a6c3d5d43a894a2c727b03c262a 2023-03-22 15:55:21,621 DEBUG [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-c9aaf840692146c29c916b1885331b30 2023-03-22 15:55:21,621 INFO [StoreOpener-b3544ae53f253dc547c7d54a0e776400-1] regionserver.HStore(310): Store=b3544ae53f253dc547c7d54a0e776400/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:21,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:21,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:21,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:55:21,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:55:21,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b3544ae53f253dc547c7d54a0e776400; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72473780, jitterRate=0.07994347810745239}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:55:21,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:21,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400., pid=101, masterSystemTime=1679500521543 2023-03-22 15:55:21,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:55:21,629 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-22 15:55:21,629 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-22 15:55:21,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-22 15:55:21,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:21,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:21,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:21,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2345ee45559d48c9feb8a4a60984e2c3, NAME => 'testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.', STARTKEY => '', ENDKEY => '1'} 2023-03-22 15:55:21,632 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=b3544ae53f253dc547c7d54a0e776400, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:21,632 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500521631"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500521631"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500521631"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500521631"}]},"ts":"1679500521631"} 2023-03-22 15:55:21,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:21,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:21,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:21,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:21,635 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-03-22 15:55:21,635 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure b3544ae53f253dc547c7d54a0e776400, server=jenkins-hbase4.apache.org,38637,1679500416185 in 242 msec 2023-03-22 15:55:21,636 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b3544ae53f253dc547c7d54a0e776400, ASSIGN in 401 msec 2023-03-22 15:55:21,640 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-22 15:55:21,640 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1912): b3544ae53f253dc547c7d54a0e776400/family1 is initiating minor compaction (all files) 2023-03-22 15:55:21,640 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b3544ae53f253dc547c7d54a0e776400/family1 in testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:21,640 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-f9075a905dd8452c8f347a85a0de825f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-b8feeb8b3e8c416882ec0c124ebd0f4d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-043c5cd69fe94f4bb70b8fbff9329fee, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-19d38f1e5b9a417fbe004d4965abcf7c] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp, totalSize=21.6 K 2023-03-22 15:55:21,641 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:21,642 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-22 15:55:21,642 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HStore(1912): b3544ae53f253dc547c7d54a0e776400/family2 is initiating minor compaction (all files) 2023-03-22 15:55:21,642 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b3544ae53f253dc547c7d54a0e776400/family2 in testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:55:21,642 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-f9075a905dd8452c8f347a85a0de825f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499420 2023-03-22 15:55:21,642 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-7abb4a6c3d5d43a894a2c727b03c262a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-70e5363498bd448089cef77631a32139, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-c9aaf840692146c29c916b1885331b30, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-18e0114e20734d5c87b318ace88eb8a2] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp, totalSize=20.4 K 2023-03-22 15:55:21,644 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1 2023-03-22 15:55:21,644 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1 2023-03-22 15:55:21,644 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-b8feeb8b3e8c416882ec0c124ebd0f4d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:21,645 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-7abb4a6c3d5d43a894a2c727b03c262a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499420 2023-03-22 15:55:21,645 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-043c5cd69fe94f4bb70b8fbff9329fee, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:21,646 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-70e5363498bd448089cef77631a32139, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:21,646 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-19d38f1e5b9a417fbe004d4965abcf7c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:21,646 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-c9aaf840692146c29c916b1885331b30, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:21,648 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2345ee45559d48c9feb8a4a60984e2c3 columnFamilyName family1 2023-03-22 15:55:21,652 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-18e0114e20734d5c87b318ace88eb8a2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:21,678 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-16f191502d2a4ad2bd45864da26c4836 2023-03-22 15:55:21,707 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-7876950a8aaf4d5eacc03c89aba69af3 2023-03-22 15:55:21,738 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] throttle.PressureAwareThroughputController(145): b3544ae53f253dc547c7d54a0e776400#family2#compaction#114 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:55:21,739 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ce9c7293befc4ac2b83081d67235637f 2023-03-22 15:55:21,764 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): b3544ae53f253dc547c7d54a0e776400#family1#compaction#115 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:55:21,769 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-f3b4e4f5037842a393873a3f040dc476 2023-03-22 15:55:21,769 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(310): Store=2345ee45559d48c9feb8a4a60984e2c3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:21,770 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:21,771 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2 2023-03-22 15:55:21,771 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2 2023-03-22 15:55:21,772 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2345ee45559d48c9feb8a4a60984e2c3 columnFamilyName family2 2023-03-22 15:55:21,780 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-71885845d5394c4f8c6fa2cf66aee993 2023-03-22 15:55:21,793 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-9f3ebcd7510d453d968b5291f038a60a 2023-03-22 15:55:21,802 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-a17d54ef6c504ff5aa86672257e08710 2023-03-22 15:55:21,810 DEBUG [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ae4e7e46dd414d1794011c71e8b2d40d 2023-03-22 15:55:21,810 INFO [StoreOpener-2345ee45559d48c9feb8a4a60984e2c3-1] regionserver.HStore(310): Store=2345ee45559d48c9feb8a4a60984e2c3/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:21,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:21,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:21,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:55:21,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:55:21,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2345ee45559d48c9feb8a4a60984e2c3; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67248456, jitterRate=0.002080082893371582}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:55:21,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:21,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3., pid=103, masterSystemTime=1679500521543 2023-03-22 15:55:21,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-22 15:55:21,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-22 15:55:21,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:21,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:21,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:21,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fc08f2f22b58b2bb5e4b114f03a4e725, NAME => 'testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.', STARTKEY => '1', ENDKEY => '2'} 2023-03-22 15:55:21,817 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=2345ee45559d48c9feb8a4a60984e2c3, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:21,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:21,817 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679500521817"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500521817"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500521817"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500521817"}]},"ts":"1679500521817"} 2023-03-22 15:55:21,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:21,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:21,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:21,819 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:21,820 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1 2023-03-22 15:55:21,820 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1 2023-03-22 15:55:21,820 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fc08f2f22b58b2bb5e4b114f03a4e725 columnFamilyName family1 2023-03-22 15:55:21,821 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=98 2023-03-22 15:55:21,821 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure 2345ee45559d48c9feb8a4a60984e2c3, server=jenkins-hbase4.apache.org,38637,1679500416185 in 427 msec 2023-03-22 15:55:21,822 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=2345ee45559d48c9feb8a4a60984e2c3, ASSIGN in 587 msec 2023-03-22 15:55:21,831 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-03e5d7bf4bbc4f9dba04343703393d6b 2023-03-22 15:55:21,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-22 15:55:21,839 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-7917639753e84614aefeb4c0d38d5c9d 2023-03-22 15:55:21,845 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8a275a5fcb784f13804cf407ba7a368d 2023-03-22 15:55:21,851 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a0ad7a2265a043bc80f148a22e45cb6f 2023-03-22 15:55:21,851 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(310): Store=fc08f2f22b58b2bb5e4b114f03a4e725/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:21,851 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:21,852 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2 2023-03-22 15:55:21,852 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2 2023-03-22 15:55:21,853 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fc08f2f22b58b2bb5e4b114f03a4e725 columnFamilyName family2 2023-03-22 15:55:21,861 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-2aa4237f0ab246f69abbc12410b88bba 2023-03-22 15:55:21,872 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-4dd66f50257149a69fc3e0617e774e1c 2023-03-22 15:55:21,883 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8b37dea4807341f996a5efdd042112a0 2023-03-22 15:55:21,894 DEBUG [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(539): loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a5af574092a54fc19d8d96cf59f9ec91 2023-03-22 15:55:21,894 INFO [StoreOpener-fc08f2f22b58b2bb5e4b114f03a4e725-1] regionserver.HStore(310): Store=fc08f2f22b58b2bb5e4b114f03a4e725/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:21,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:21,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:21,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:55:21,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:55:21,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fc08f2f22b58b2bb5e4b114f03a4e725; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71915538, jitterRate=0.07162502408027649}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:55:21,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:21,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725., pid=102, masterSystemTime=1679500521543 2023-03-22 15:55:21,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-22 15:55:21,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-22 15:55:21,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:21,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:21,904 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=fc08f2f22b58b2bb5e4b114f03a4e725, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:21,904 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500521903"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500521903"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500521903"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500521903"}]},"ts":"1679500521903"} 2023-03-22 15:55:21,910 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=99 2023-03-22 15:55:21,911 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=99, state=SUCCESS; OpenRegionProcedure fc08f2f22b58b2bb5e4b114f03a4e725, server=jenkins-hbase4.apache.org,38637,1679500416185 in 513 msec 2023-03-22 15:55:21,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-03-22 15:55:21,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=fc08f2f22b58b2bb5e4b114f03a4e725, ASSIGN in 676 msec 2023-03-22 15:55:21,912 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500521912"}]},"ts":"1679500521912"} 2023-03-22 15:55:21,913 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-22 15:55:21,915 INFO [PEWorker-5] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-22 15:55:21,916 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 694 msec 2023-03-22 15:55:22,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-22 15:55:22,334 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-22 15:55:22,348 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:22,362 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:22,382 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:22,383 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:22,394 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-22 15:55:22,433 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-22 15:55:22,434 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:22,434 DEBUG [Listener at localhost/43289] zookeeper.ReadOnlyZKClient(139): Connect 0x3eac4e78 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:55:22,462 DEBUG [Listener at localhost/43289] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7c7517f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:55:22,463 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:22,463 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:22,463 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:22,633 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322c33f808ca5404ced9d5edc6fa516a5fa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230322c33f808ca5404ced9d5edc6fa516a5fa 2023-03-22 15:55:22,634 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:22,647 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:55:22,653 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46022, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:55:22,661 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:55:22,676 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41554, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:55:22,686 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322/family1/9a401d78c4724a9192df311c3325613d first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-22 15:55:22,686 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=32 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322/family1/9a401d78c4724a9192df311c3325613d}] 2023-03-22 15:55:22,693 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322/family1/9a401d78c4724a9192df311c3325613d for inclusion in 2345ee45559d48c9feb8a4a60984e2c3/family1 2023-03-22 15:55:22,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-22 15:55:22,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last=1 2023-03-22 15:55:22,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:22,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322/family1/9a401d78c4724a9192df311c3325613d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__ad4os29m0sdnedn1qc6ccmku8a1a2etesusnog4mgagsg8o5kc925v5hgkgb4ga0/family1/9a401d78c4724a9192df311c3325613d 2023-03-22 15:55:22,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__ad4os29m0sdnedn1qc6ccmku8a1a2etesusnog4mgagsg8o5kc925v5hgkgb4ga0/family1/9a401d78c4724a9192df311c3325613d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/70b9883ec33c4314a8b40507c4550a5e_SeqId_34_ 2023-03-22 15:55:22,700 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__ad4os29m0sdnedn1qc6ccmku8a1a2etesusnog4mgagsg8o5kc925v5hgkgb4ga0/family1/9a401d78c4724a9192df311c3325613d into 2345ee45559d48c9feb8a4a60984e2c3/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/70b9883ec33c4314a8b40507c4550a5e_SeqId_34_ - updating store file list. 2023-03-22 15:55:22,703 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/70b9883ec33c4314a8b40507c4550a5e_SeqId_34_ into 2345ee45559d48c9feb8a4a60984e2c3/family1 2023-03-22 15:55:22,703 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__ad4os29m0sdnedn1qc6ccmku8a1a2etesusnog4mgagsg8o5kc925v5hgkgb4ga0/family1/9a401d78c4724a9192df311c3325613d into 2345ee45559d48c9feb8a4a60984e2c3/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/70b9883ec33c4314a8b40507c4550a5e_SeqId_34_) 2023-03-22 15:55:22,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__ad4os29m0sdnedn1qc6ccmku8a1a2etesusnog4mgagsg8o5kc925v5hgkgb4ga0/family1/9a401d78c4724a9192df311c3325613d 2023-03-22 15:55:22,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:22,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-22 15:55:22,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(1912): 2345ee45559d48c9feb8a4a60984e2c3/family1 is initiating minor compaction (all files) 2023-03-22 15:55:22,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@10a4f8c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-22 15:55:22,709 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:22,709 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:22,711 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230322127ec934bcb24f44b0fdfe9f5c430400 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230322127ec934bcb24f44b0fdfe9f5c430400 2023-03-22 15:55:22,712 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303227142606f287940e88cf21ec6c6677b0e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303227142606f287940e88cf21ec6c6677b0e 2023-03-22 15:55:22,714 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303229ff89dda9c844b89bb8dd67fe572459c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303229ff89dda9c844b89bb8dd67fe572459c 2023-03-22 15:55:22,715 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230322f18742089e924f019e0769cd09b681ed to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230322f18742089e924f019e0769cd09b681ed 2023-03-22 15:55:22,715 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-22 15:55:22,715 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230322' 2023-03-22 15:55:22,715 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:23,016 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322e5e7a2f5f66047c69c9b9f60c8443962 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230322e5e7a2f5f66047c69c9b9f60c8443962 2023-03-22 15:55:23,017 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:23,029 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family2/d32e58493f9b4007878b56bbcd40ce3e as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/d32e58493f9b4007878b56bbcd40ce3e 2023-03-22 15:55:23,031 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322f7bc3936b7bc4700adf297da16ca6b3b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230322f7bc3936b7bc4700adf297da16ca6b3b 2023-03-22 15:55:23,033 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:23,035 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-22 15:55:23,036 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in b3544ae53f253dc547c7d54a0e776400/family2 of b3544ae53f253dc547c7d54a0e776400 into d32e58493f9b4007878b56bbcd40ce3e(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:23,036 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:23,036 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400., storeName=b3544ae53f253dc547c7d54a0e776400/family2, priority=12, startTime=1679500521629; duration=1sec 2023-03-22 15:55:23,036 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-03-22 15:55:23,036 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2345ee45559d48c9feb8a4a60984e2c3/family1 in testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:23,036 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ce9c7293befc4ac2b83081d67235637f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-f3b4e4f5037842a393873a3f040dc476, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-7876950a8aaf4d5eacc03c89aba69af3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-16f191502d2a4ad2bd45864da26c4836, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/70b9883ec33c4314a8b40507c4550a5e_SeqId_34_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp, totalSize=28.9 K 2023-03-22 15:55:23,039 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/.tmp/family1/bb7ac2a55eba46ea8d0754ff087059de as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/bb7ac2a55eba46ea8d0754ff087059de 2023-03-22 15:55:23,040 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ce9c7293befc4ac2b83081d67235637f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499417 2023-03-22 15:55:23,041 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-f3b4e4f5037842a393873a3f040dc476, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:23,042 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-7876950a8aaf4d5eacc03c89aba69af3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:23,054 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-16f191502d2a4ad2bd45864da26c4836, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:23,055 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting 70b9883ec33c4314a8b40507c4550a5e_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1679500499417 2023-03-22 15:55:23,069 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in b3544ae53f253dc547c7d54a0e776400/family1 of b3544ae53f253dc547c7d54a0e776400 into bb7ac2a55eba46ea8d0754ff087059de(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:23,069 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:23,069 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400., storeName=b3544ae53f253dc547c7d54a0e776400/family1, priority=12, startTime=1679500521629; duration=1sec 2023-03-22 15:55:23,069 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-22 15:55:23,069 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-22 15:55:23,073 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322/family1/0a28d7f84ecd4013b7ea6d1b09f60527 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-22 15:55:23,074 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322/family1/0a28d7f84ecd4013b7ea6d1b09f60527}] 2023-03-22 15:55:23,077 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-22 15:55:23,077 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322/family1/0a28d7f84ecd4013b7ea6d1b09f60527 for inclusion in b3544ae53f253dc547c7d54a0e776400/family1 2023-03-22 15:55:23,077 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1912): 2345ee45559d48c9feb8a4a60984e2c3/family2 is initiating minor compaction (all files) 2023-03-22 15:55:23,078 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2345ee45559d48c9feb8a4a60984e2c3/family2 in testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:55:23,078 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-71885845d5394c4f8c6fa2cf66aee993, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-a17d54ef6c504ff5aa86672257e08710, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ae4e7e46dd414d1794011c71e8b2d40d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-9f3ebcd7510d453d968b5291f038a60a] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp, totalSize=20.4 K 2023-03-22 15:55:23,079 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-71885845d5394c4f8c6fa2cf66aee993, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499417 2023-03-22 15:55:23,080 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-a17d54ef6c504ff5aa86672257e08710, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:23,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-22 15:55:23,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-22 15:55:23,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:55:23,082 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ae4e7e46dd414d1794011c71e8b2d40d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:23,083 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-9f3ebcd7510d453d968b5291f038a60a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:23,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322/family1/0a28d7f84ecd4013b7ea6d1b09f60527 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__fdsl188j3rem6r78f9e862stfcchoo4pdfskhcak67ma2neu9q7o6g12tkfp3avg/family1/0a28d7f84ecd4013b7ea6d1b09f60527 2023-03-22 15:55:23,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__fdsl188j3rem6r78f9e862stfcchoo4pdfskhcak67ma2neu9q7o6g12tkfp3avg/family1/0a28d7f84ecd4013b7ea6d1b09f60527 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/5a61ac5389b746c5b5138b5a3bf8ea3c_SeqId_36_ 2023-03-22 15:55:23,094 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__fdsl188j3rem6r78f9e862stfcchoo4pdfskhcak67ma2neu9q7o6g12tkfp3avg/family1/0a28d7f84ecd4013b7ea6d1b09f60527 into b3544ae53f253dc547c7d54a0e776400/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/5a61ac5389b746c5b5138b5a3bf8ea3c_SeqId_36_ - updating store file list. 2023-03-22 15:55:23,100 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322/family1/d3b53a8a50ee49a78d431c8c0790a584 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-22 15:55:23,101 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322/family1/d3b53a8a50ee49a78d431c8c0790a584}] 2023-03-22 15:55:23,102 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/5a61ac5389b746c5b5138b5a3bf8ea3c_SeqId_36_ into b3544ae53f253dc547c7d54a0e776400/family1 2023-03-22 15:55:23,102 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__fdsl188j3rem6r78f9e862stfcchoo4pdfskhcak67ma2neu9q7o6g12tkfp3avg/family1/0a28d7f84ecd4013b7ea6d1b09f60527 into b3544ae53f253dc547c7d54a0e776400/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/5a61ac5389b746c5b5138b5a3bf8ea3c_SeqId_36_) 2023-03-22 15:55:23,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__fdsl188j3rem6r78f9e862stfcchoo4pdfskhcak67ma2neu9q7o6g12tkfp3avg/family1/0a28d7f84ecd4013b7ea6d1b09f60527 2023-03-22 15:55:23,109 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] throttle.PressureAwareThroughputController(145): 2345ee45559d48c9feb8a4a60984e2c3#family1#compaction#116 average throughput is 0.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:55:23,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-f9075a905dd8452c8f347a85a0de825f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-b8feeb8b3e8c416882ec0c124ebd0f4d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-043c5cd69fe94f4bb70b8fbff9329fee, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-19d38f1e5b9a417fbe004d4965abcf7c] to archive 2023-03-22 15:55:23,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322/family1/d3b53a8a50ee49a78d431c8c0790a584 for inclusion in fc08f2f22b58b2bb5e4b114f03a4e725/family1 2023-03-22 15:55:23,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:23,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-22 15:55:23,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-22 15:55:23,122 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:23,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:23,123 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:23,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322/family1/d3b53a8a50ee49a78d431c8c0790a584 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__8i02f6jbkeavea0i4nracujut8fh0tb2ild15hhvf37pisi6h48nigfnftjruinn/family1/d3b53a8a50ee49a78d431c8c0790a584 2023-03-22 15:55:23,127 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303220495f367c8bf4559a0ce2059d3c42779 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303220495f367c8bf4559a0ce2059d3c42779 2023-03-22 15:55:23,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-f9075a905dd8452c8f347a85a0de825f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-f9075a905dd8452c8f347a85a0de825f 2023-03-22 15:55:23,128 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230322059296d521f74be3822284aae2c125ab to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230322059296d521f74be3822284aae2c125ab 2023-03-22 15:55:23,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-b8feeb8b3e8c416882ec0c124ebd0f4d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-b8feeb8b3e8c416882ec0c124ebd0f4d 2023-03-22 15:55:23,130 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032262ef361c18a54c10b19099f1a899c95e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032262ef361c18a54c10b19099f1a899c95e 2023-03-22 15:55:23,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__8i02f6jbkeavea0i4nracujut8fh0tb2ild15hhvf37pisi6h48nigfnftjruinn/family1/d3b53a8a50ee49a78d431c8c0790a584 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3fff88d7681d43fda9ec6396b980e8f7_SeqId_34_ 2023-03-22 15:55:23,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__8i02f6jbkeavea0i4nracujut8fh0tb2ild15hhvf37pisi6h48nigfnftjruinn/family1/d3b53a8a50ee49a78d431c8c0790a584 into fc08f2f22b58b2bb5e4b114f03a4e725/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3fff88d7681d43fda9ec6396b980e8f7_SeqId_34_ - updating store file list. 2023-03-22 15:55:23,130 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-043c5cd69fe94f4bb70b8fbff9329fee to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-043c5cd69fe94f4bb70b8fbff9329fee 2023-03-22 15:55:23,131 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303227dae9615fad648e6b55bc956feb5a1e4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303227dae9615fad648e6b55bc956feb5a1e4 2023-03-22 15:55:23,131 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-22 15:55:23,131 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230322' 2023-03-22 15:55:23,131 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:23,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-19d38f1e5b9a417fbe004d4965abcf7c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family1/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-19d38f1e5b9a417fbe004d4965abcf7c 2023-03-22 15:55:23,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-7abb4a6c3d5d43a894a2c727b03c262a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-70e5363498bd448089cef77631a32139, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-c9aaf840692146c29c916b1885331b30, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-18e0114e20734d5c87b318ace88eb8a2] to archive 2023-03-22 15:55:23,135 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:23,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-7abb4a6c3d5d43a894a2c727b03c262a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-7abb4a6c3d5d43a894a2c727b03c262a 2023-03-22 15:55:23,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3fff88d7681d43fda9ec6396b980e8f7_SeqId_34_ into fc08f2f22b58b2bb5e4b114f03a4e725/family1 2023-03-22 15:55:23,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__8i02f6jbkeavea0i4nracujut8fh0tb2ild15hhvf37pisi6h48nigfnftjruinn/family1/d3b53a8a50ee49a78d431c8c0790a584 into fc08f2f22b58b2bb5e4b114f03a4e725/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3fff88d7681d43fda9ec6396b980e8f7_SeqId_34_) 2023-03-22 15:55:23,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testCompactionWithHFileLink__8i02f6jbkeavea0i4nracujut8fh0tb2ild15hhvf37pisi6h48nigfnftjruinn/family1/d3b53a8a50ee49a78d431c8c0790a584 2023-03-22 15:55:23,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:23,139 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2345ee45559d48c9feb8a4a60984e2c3#family2#compaction#117 average throughput is 0.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:55:23,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-70e5363498bd448089cef77631a32139 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-70e5363498bd448089cef77631a32139 2023-03-22 15:55:23,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-c9aaf840692146c29c916b1885331b30 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-c9aaf840692146c29c916b1885331b30 2023-03-22 15:55:23,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-18e0114e20734d5c87b318ace88eb8a2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/family2/testCompactionWithHFileLink=b3544ae53f253dc547c7d54a0e776400-18e0114e20734d5c87b318ace88eb8a2 2023-03-22 15:55:23,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-22 15:55:23,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(1912): fc08f2f22b58b2bb5e4b114f03a4e725/family1 is initiating minor compaction (all files) 2023-03-22 15:55:23,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@67f0db0a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-22 15:55:23,152 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230322 table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:23,153 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:23,154 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023032292092a0688b74b8292c4011b93e1193a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023032292092a0688b74b8292c4011b93e1193a 2023-03-22 15:55:23,155 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230322b5802331476e4c4d8d93c74275b69e33 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230322b5802331476e4c4d8d93c74275b69e33 2023-03-22 15:55:23,156 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230322c616bcebc940418b8c1df3a672466bc6 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230322c616bcebc940418b8c1df3a672466bc6 2023-03-22 15:55:23,158 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230322e5039a9c9c6e462ea98ef70511033233 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230322e5039a9c9c6e462ea98ef70511033233 2023-03-22 15:55:23,158 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-22 15:55:23,158 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230322' 2023-03-22 15:55:23,158 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230322 for table='testCompactionWithHFileLink' and column='family1' 2023-03-22 15:55:23,158 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-22 15:55:23,161 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:23,163 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:23,169 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:23,170 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:23,177 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:23,178 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:23,196 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=285 (was 261) - Thread LEAK? -, OpenFileDescriptor=762 (was 644) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=364 (was 403), ProcessCount=170 (was 170), AvailableMemoryMB=6298 (was 6478) 2023-03-22 15:55:23,212 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=285, OpenFileDescriptor=762, MaxFileDescriptor=60000, SystemLoadAverage=364, ProcessCount=170, AvailableMemoryMB=6297 2023-03-22 15:55:23,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-22 15:55:23,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-22 15:55:23,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-22 15:55:23,225 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-22 15:55:23,227 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 8 msec 2023-03-22 15:55:23,296 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:55:23,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-22 15:55:23,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-22 15:55:23,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-22 15:55:23,328 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:55:23,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-22 15:55:23,329 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:55:23,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-22 15:55:23,332 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:23,332 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:23,332 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:23,333 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164 empty. 2023-03-22 15:55:23,333 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43 empty. 2023-03-22 15:55:23,333 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:23,333 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:23,334 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0 empty. 2023-03-22 15:55:23,335 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:23,335 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-22 15:55:23,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-22 15:55:23,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-22 15:55:23,750 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-22 15:55:23,752 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 2c6ba5db98f0e0042380c0a57cebc9d0, NAME => 'ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:55:23,752 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 9ae79c6df98f35ced4ccb3fbc1d46164, NAME => 'ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:55:23,752 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 23b8dc9b2c9ae9fdf9b83a4c4302cb43, NAME => 'ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:55:23,772 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:23,772 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 9ae79c6df98f35ced4ccb3fbc1d46164, disabling compactions & flushes 2023-03-22 15:55:23,772 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:23,772 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:23,772 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. after waiting 0 ms 2023-03-22 15:55:23,772 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:23,772 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:23,772 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:23,773 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:23,773 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 23b8dc9b2c9ae9fdf9b83a4c4302cb43, disabling compactions & flushes 2023-03-22 15:55:23,773 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:23,773 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:23,773 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. after waiting 0 ms 2023-03-22 15:55:23,774 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:23,774 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:23,774 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:23,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-22 15:55:23,960 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family1/cb9d369cf495447e994cc376eba41297 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/cb9d369cf495447e994cc376eba41297 2023-03-22 15:55:23,967 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 2345ee45559d48c9feb8a4a60984e2c3/family1 of 2345ee45559d48c9feb8a4a60984e2c3 into cb9d369cf495447e994cc376eba41297(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:23,967 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:23,967 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1679500522707; duration=0sec 2023-03-22 15:55:23,968 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-22 15:55:23,968 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegion(2259): Starting compaction of fc08f2f22b58b2bb5e4b114f03a4e725/family1 in testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:23,968 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-7917639753e84614aefeb4c0d38d5c9d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-03e5d7bf4bbc4f9dba04343703393d6b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a0ad7a2265a043bc80f148a22e45cb6f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8a275a5fcb784f13804cf407ba7a368d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3fff88d7681d43fda9ec6396b980e8f7_SeqId_34_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp, totalSize=28.9 K 2023-03-22 15:55:23,969 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-7917639753e84614aefeb4c0d38d5c9d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499419 2023-03-22 15:55:23,969 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-03e5d7bf4bbc4f9dba04343703393d6b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:23,970 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a0ad7a2265a043bc80f148a22e45cb6f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:23,970 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8a275a5fcb784f13804cf407ba7a368d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:23,971 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.Compactor(207): Compacting 3fff88d7681d43fda9ec6396b980e8f7_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1679500499419 2023-03-22 15:55:23,973 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/.tmp/family2/7b53f1592a2442c1abf5de79dd09bc7b as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/7b53f1592a2442c1abf5de79dd09bc7b 2023-03-22 15:55:23,977 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 2345ee45559d48c9feb8a4a60984e2c3/family2 of 2345ee45559d48c9feb8a4a60984e2c3 into 7b53f1592a2442c1abf5de79dd09bc7b(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:23,977 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:55:23,978 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3., storeName=2345ee45559d48c9feb8a4a60984e2c3/family2, priority=12, startTime=1679500521815; duration=0sec 2023-03-22 15:55:23,978 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-22 15:55:23,978 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 5 compacting, 0 eligible, 16 blocking 2023-03-22 15:55:23,978 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-22 15:55:23,978 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-22 15:55:23,978 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. because compaction request was cancelled 2023-03-22 15:55:23,978 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-22 15:55:23,981 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-22 15:55:23,981 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1912): fc08f2f22b58b2bb5e4b114f03a4e725/family2 is initiating minor compaction (all files) 2023-03-22 15:55:23,981 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fc08f2f22b58b2bb5e4b114f03a4e725/family2 in testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:55:23,981 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-2aa4237f0ab246f69abbc12410b88bba, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-4dd66f50257149a69fc3e0617e774e1c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8b37dea4807341f996a5efdd042112a0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a5af574092a54fc19d8d96cf59f9ec91] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp, totalSize=20.4 K 2023-03-22 15:55:23,982 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-2aa4237f0ab246f69abbc12410b88bba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500499419 2023-03-22 15:55:23,983 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-4dd66f50257149a69fc3e0617e774e1c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500501211 2023-03-22 15:55:23,984 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8b37dea4807341f996a5efdd042112a0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500502616 2023-03-22 15:55:23,985 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a5af574092a54fc19d8d96cf59f9ec91, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500504413 2023-03-22 15:55:23,995 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] throttle.PressureAwareThroughputController(145): fc08f2f22b58b2bb5e4b114f03a4e725#family1#compaction#118 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:55:24,012 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ce9c7293befc4ac2b83081d67235637f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-f3b4e4f5037842a393873a3f040dc476, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-7876950a8aaf4d5eacc03c89aba69af3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-16f191502d2a4ad2bd45864da26c4836, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/70b9883ec33c4314a8b40507c4550a5e_SeqId_34_] to archive 2023-03-22 15:55:24,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:24,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ce9c7293befc4ac2b83081d67235637f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ce9c7293befc4ac2b83081d67235637f 2023-03-22 15:55:24,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-f3b4e4f5037842a393873a3f040dc476 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-f3b4e4f5037842a393873a3f040dc476 2023-03-22 15:55:24,017 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-7876950a8aaf4d5eacc03c89aba69af3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-7876950a8aaf4d5eacc03c89aba69af3 2023-03-22 15:55:24,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-16f191502d2a4ad2bd45864da26c4836 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-16f191502d2a4ad2bd45864da26c4836 2023-03-22 15:55:24,020 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): fc08f2f22b58b2bb5e4b114f03a4e725#family2#compaction#119 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 80.77 MB/second 2023-03-22 15:55:24,021 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/70b9883ec33c4314a8b40507c4550a5e_SeqId_34_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family1/70b9883ec33c4314a8b40507c4550a5e_SeqId_34_ 2023-03-22 15:55:24,023 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-71885845d5394c4f8c6fa2cf66aee993, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-a17d54ef6c504ff5aa86672257e08710, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ae4e7e46dd414d1794011c71e8b2d40d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-9f3ebcd7510d453d968b5291f038a60a] to archive 2023-03-22 15:55:24,025 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:24,027 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-71885845d5394c4f8c6fa2cf66aee993 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-71885845d5394c4f8c6fa2cf66aee993 2023-03-22 15:55:24,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-a17d54ef6c504ff5aa86672257e08710 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-a17d54ef6c504ff5aa86672257e08710 2023-03-22 15:55:24,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ae4e7e46dd414d1794011c71e8b2d40d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-ae4e7e46dd414d1794011c71e8b2d40d 2023-03-22 15:55:24,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-9f3ebcd7510d453d968b5291f038a60a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/family2/testCompactionWithHFileLink=2345ee45559d48c9feb8a4a60984e2c3-9f3ebcd7510d453d968b5291f038a60a 2023-03-22 15:55:24,166 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:24,166 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 2c6ba5db98f0e0042380c0a57cebc9d0, disabling compactions & flushes 2023-03-22 15:55:24,166 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:24,166 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:24,166 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. after waiting 0 ms 2023-03-22 15:55:24,166 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:24,166 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:24,166 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:24,169 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:55:24,170 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679500524170"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500524170"}]},"ts":"1679500524170"} 2023-03-22 15:55:24,170 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679500524170"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500524170"}]},"ts":"1679500524170"} 2023-03-22 15:55:24,170 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679500524170"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500524170"}]},"ts":"1679500524170"} 2023-03-22 15:55:24,172 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-22 15:55:24,173 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:55:24,173 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500524173"}]},"ts":"1679500524173"} 2023-03-22 15:55:24,174 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-22 15:55:24,178 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9ae79c6df98f35ced4ccb3fbc1d46164, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=23b8dc9b2c9ae9fdf9b83a4c4302cb43, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2c6ba5db98f0e0042380c0a57cebc9d0, ASSIGN}] 2023-03-22 15:55:24,180 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2c6ba5db98f0e0042380c0a57cebc9d0, ASSIGN 2023-03-22 15:55:24,180 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=23b8dc9b2c9ae9fdf9b83a4c4302cb43, ASSIGN 2023-03-22 15:55:24,180 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9ae79c6df98f35ced4ccb3fbc1d46164, ASSIGN 2023-03-22 15:55:24,180 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2c6ba5db98f0e0042380c0a57cebc9d0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:55:24,180 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=23b8dc9b2c9ae9fdf9b83a4c4302cb43, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:55:24,180 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9ae79c6df98f35ced4ccb3fbc1d46164, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:55:24,333 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=23b8dc9b2c9ae9fdf9b83a4c4302cb43, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:24,333 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9ae79c6df98f35ced4ccb3fbc1d46164, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:24,333 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=2c6ba5db98f0e0042380c0a57cebc9d0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:24,333 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679500524333"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500524333"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500524333"}]},"ts":"1679500524333"} 2023-03-22 15:55:24,333 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679500524333"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500524333"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500524333"}]},"ts":"1679500524333"} 2023-03-22 15:55:24,333 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679500524333"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500524333"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500524333"}]},"ts":"1679500524333"} 2023-03-22 15:55:24,340 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure 23b8dc9b2c9ae9fdf9b83a4c4302cb43, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:24,340 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure 9ae79c6df98f35ced4ccb3fbc1d46164, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:24,341 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure 2c6ba5db98f0e0042380c0a57cebc9d0, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:24,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-22 15:55:24,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:24,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2c6ba5db98f0e0042380c0a57cebc9d0, NAME => 'ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0.', STARTKEY => '2', ENDKEY => ''} 2023-03-22 15:55:24,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:24,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:24,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:24,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:24,495 INFO [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:24,496 DEBUG [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1 2023-03-22 15:55:24,496 DEBUG [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1 2023-03-22 15:55:24,497 INFO [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c6ba5db98f0e0042380c0a57cebc9d0 columnFamilyName family1 2023-03-22 15:55:24,498 INFO [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] regionserver.HStore(310): Store=2c6ba5db98f0e0042380c0a57cebc9d0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:24,498 INFO [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:24,499 DEBUG [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2 2023-03-22 15:55:24,499 DEBUG [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2 2023-03-22 15:55:24,500 INFO [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c6ba5db98f0e0042380c0a57cebc9d0 columnFamilyName family2 2023-03-22 15:55:24,501 INFO [StoreOpener-2c6ba5db98f0e0042380c0a57cebc9d0-1] regionserver.HStore(310): Store=2c6ba5db98f0e0042380c0a57cebc9d0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:24,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:24,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:24,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:55:24,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:55:24,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:55:24,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2c6ba5db98f0e0042380c0a57cebc9d0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66038170, jitterRate=-0.015954583883285522}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:55:24,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:24,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0., pid=111, masterSystemTime=1679500524491 2023-03-22 15:55:24,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:24,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:24,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:24,509 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=2c6ba5db98f0e0042380c0a57cebc9d0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:24,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 23b8dc9b2c9ae9fdf9b83a4c4302cb43, NAME => 'ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43.', STARTKEY => '1', ENDKEY => '2'} 2023-03-22 15:55:24,509 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679500524509"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500524509"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500524509"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500524509"}]},"ts":"1679500524509"} 2023-03-22 15:55:24,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:24,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:24,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:24,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:24,512 INFO [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:24,512 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-03-22 15:55:24,512 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure 2c6ba5db98f0e0042380c0a57cebc9d0, server=jenkins-hbase4.apache.org,38637,1679500416185 in 170 msec 2023-03-22 15:55:24,513 DEBUG [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1 2023-03-22 15:55:24,513 DEBUG [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1 2023-03-22 15:55:24,514 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2c6ba5db98f0e0042380c0a57cebc9d0, ASSIGN in 334 msec 2023-03-22 15:55:24,514 INFO [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 23b8dc9b2c9ae9fdf9b83a4c4302cb43 columnFamilyName family1 2023-03-22 15:55:24,516 INFO [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] regionserver.HStore(310): Store=23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:24,516 INFO [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:24,517 DEBUG [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2 2023-03-22 15:55:24,517 DEBUG [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2 2023-03-22 15:55:24,518 INFO [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 23b8dc9b2c9ae9fdf9b83a4c4302cb43 columnFamilyName family2 2023-03-22 15:55:24,519 INFO [StoreOpener-23b8dc9b2c9ae9fdf9b83a4c4302cb43-1] regionserver.HStore(310): Store=23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:24,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:24,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:24,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:55:24,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:55:24,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:55:24,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 23b8dc9b2c9ae9fdf9b83a4c4302cb43; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69846127, jitterRate=0.04078839719295502}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:55:24,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:24,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43., pid=109, masterSystemTime=1679500524491 2023-03-22 15:55:24,531 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=23b8dc9b2c9ae9fdf9b83a4c4302cb43, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:24,531 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679500524531"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500524531"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500524531"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500524531"}]},"ts":"1679500524531"} 2023-03-22 15:55:24,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:24,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:24,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:24,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9ae79c6df98f35ced4ccb3fbc1d46164, NAME => 'ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164.', STARTKEY => '', ENDKEY => '1'} 2023-03-22 15:55:24,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:24,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:24,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:24,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:24,533 INFO [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:24,534 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=107 2023-03-22 15:55:24,534 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure 23b8dc9b2c9ae9fdf9b83a4c4302cb43, server=jenkins-hbase4.apache.org,38637,1679500416185 in 192 msec 2023-03-22 15:55:24,535 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=23b8dc9b2c9ae9fdf9b83a4c4302cb43, ASSIGN in 356 msec 2023-03-22 15:55:24,536 DEBUG [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1 2023-03-22 15:55:24,536 DEBUG [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1 2023-03-22 15:55:24,537 INFO [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ae79c6df98f35ced4ccb3fbc1d46164 columnFamilyName family1 2023-03-22 15:55:24,538 INFO [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] regionserver.HStore(310): Store=9ae79c6df98f35ced4ccb3fbc1d46164/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:24,539 INFO [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:24,540 DEBUG [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2 2023-03-22 15:55:24,540 DEBUG [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2 2023-03-22 15:55:24,540 INFO [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ae79c6df98f35ced4ccb3fbc1d46164 columnFamilyName family2 2023-03-22 15:55:24,541 INFO [StoreOpener-9ae79c6df98f35ced4ccb3fbc1d46164-1] regionserver.HStore(310): Store=9ae79c6df98f35ced4ccb3fbc1d46164/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:24,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:24,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:24,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:55:24,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:55:24,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:55:24,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9ae79c6df98f35ced4ccb3fbc1d46164; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63371501, jitterRate=-0.05569104850292206}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:55:24,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:24,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164., pid=110, masterSystemTime=1679500524491 2023-03-22 15:55:24,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:24,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:24,551 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9ae79c6df98f35ced4ccb3fbc1d46164, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:24,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679500524550"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500524550"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500524550"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500524550"}]},"ts":"1679500524550"} 2023-03-22 15:55:24,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-03-22 15:55:24,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure 9ae79c6df98f35ced4ccb3fbc1d46164, server=jenkins-hbase4.apache.org,38637,1679500416185 in 212 msec 2023-03-22 15:55:24,555 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-03-22 15:55:24,555 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9ae79c6df98f35ced4ccb3fbc1d46164, ASSIGN in 375 msec 2023-03-22 15:55:24,555 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:55:24,555 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500524555"}]},"ts":"1679500524555"} 2023-03-22 15:55:24,556 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-22 15:55:24,558 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:55:24,561 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2320 sec 2023-03-22 15:55:24,844 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family1/238374b1ecb84eafaad1dd31af5c67e1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/238374b1ecb84eafaad1dd31af5c67e1 2023-03-22 15:55:24,849 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in fc08f2f22b58b2bb5e4b114f03a4e725/family1 of fc08f2f22b58b2bb5e4b114f03a4e725 into 238374b1ecb84eafaad1dd31af5c67e1(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:24,849 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:24,849 INFO [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1679500523147; duration=0sec 2023-03-22 15:55:24,850 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-22 15:55:24,850 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-03-22 15:55:24,850 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-22 15:55:24,850 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-22 15:55:24,850 DEBUG [RS:0;jenkins-hbase4:38637-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. because compaction request was cancelled 2023-03-22 15:55:24,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-7917639753e84614aefeb4c0d38d5c9d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-03e5d7bf4bbc4f9dba04343703393d6b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a0ad7a2265a043bc80f148a22e45cb6f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8a275a5fcb784f13804cf407ba7a368d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3fff88d7681d43fda9ec6396b980e8f7_SeqId_34_] to archive 2023-03-22 15:55:24,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:24,916 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-7917639753e84614aefeb4c0d38d5c9d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-7917639753e84614aefeb4c0d38d5c9d 2023-03-22 15:55:24,918 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-03e5d7bf4bbc4f9dba04343703393d6b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-03e5d7bf4bbc4f9dba04343703393d6b 2023-03-22 15:55:24,919 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a0ad7a2265a043bc80f148a22e45cb6f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a0ad7a2265a043bc80f148a22e45cb6f 2023-03-22 15:55:24,920 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8a275a5fcb784f13804cf407ba7a368d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8a275a5fcb784f13804cf407ba7a368d 2023-03-22 15:55:24,921 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3fff88d7681d43fda9ec6396b980e8f7_SeqId_34_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family1/3fff88d7681d43fda9ec6396b980e8f7_SeqId_34_ 2023-03-22 15:55:25,255 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/.tmp/family2/488bc7ae9e99492d997aa59a0a500127 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/488bc7ae9e99492d997aa59a0a500127 2023-03-22 15:55:25,260 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in fc08f2f22b58b2bb5e4b114f03a4e725/family2 of fc08f2f22b58b2bb5e4b114f03a4e725 into 488bc7ae9e99492d997aa59a0a500127(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:25,260 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:55:25,260 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725., storeName=fc08f2f22b58b2bb5e4b114f03a4e725/family2, priority=12, startTime=1679500521902; duration=1sec 2023-03-22 15:55:25,260 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:55:25,311 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-2aa4237f0ab246f69abbc12410b88bba, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-4dd66f50257149a69fc3e0617e774e1c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8b37dea4807341f996a5efdd042112a0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a5af574092a54fc19d8d96cf59f9ec91] to archive 2023-03-22 15:55:25,312 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:25,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-2aa4237f0ab246f69abbc12410b88bba to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-2aa4237f0ab246f69abbc12410b88bba 2023-03-22 15:55:25,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-4dd66f50257149a69fc3e0617e774e1c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-4dd66f50257149a69fc3e0617e774e1c 2023-03-22 15:55:25,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8b37dea4807341f996a5efdd042112a0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-8b37dea4807341f996a5efdd042112a0 2023-03-22 15:55:25,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a5af574092a54fc19d8d96cf59f9ec91 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/family2/testCompactionWithHFileLink=fc08f2f22b58b2bb5e4b114f03a4e725-a5af574092a54fc19d8d96cf59f9ec91 2023-03-22 15:55:25,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-22 15:55:25,435 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-22 15:55:25,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:25,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:25,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:25,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:25,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-22 15:55:25,444 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-22 15:55:25,444 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:25,444 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-22 15:55:25,444 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-22 15:55:25,445 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:25,445 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:25,446 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:25,446 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:25,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:25,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:25,446 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:25,446 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:25,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:25,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:25,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:25,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:25,447 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-22 15:55:25,447 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-22 15:55:25,447 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-22 15:55:25,447 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:25,447 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-22 15:55:25,447 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:25,447 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:25,447 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:25,448 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:25,448 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. started... 2023-03-22 15:55:25,448 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:25,448 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. started... 2023-03-22 15:55:25,448 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. started... 2023-03-22 15:55:25,449 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-22 15:55:25,450 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ae79c6df98f35ced4ccb3fbc1d46164 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-22 15:55:25,450 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2c6ba5db98f0e0042380c0a57cebc9d0 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-22 15:55:25,867 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322eb8f8b9ef71b423ea8271852764ddb96 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230322eb8f8b9ef71b423ea8271852764ddb96 2023-03-22 15:55:25,868 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322825e7205812e4d02922e79029758e82d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230322825e7205812e4d02922e79029758e82d 2023-03-22 15:55:25,868 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303222967290bb8564a7182e2128072da685e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303222967290bb8564a7182e2128072da685e 2023-03-22 15:55:25,968 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-22 15:55:26,273 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/7a1ee5b341654214b0149e8895c86546 2023-03-22 15:55:26,284 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/2180a6d9f53a4db1ba6801549d1c6916 2023-03-22 15:55:26,286 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/30cb8b2d9a1a49a6af5e7e9b8c2cc2f2 2023-03-22 15:55:26,685 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303223db96e9c53be4702b19560c94730b328 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303223db96e9c53be4702b19560c94730b328 2023-03-22 15:55:26,696 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303224daca3f4f1f24b30a97e3b5032006856 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303224daca3f4f1f24b30a97e3b5032006856 2023-03-22 15:55:26,698 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322faae7fe99f3442da86c0c6a28edf7075 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230322faae7fe99f3442da86c0c6a28edf7075 2023-03-22 15:55:27,090 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/d98c62e9217445fca22c34657d04df70 2023-03-22 15:55:27,094 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/7a1ee5b341654214b0149e8895c86546 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/7a1ee5b341654214b0149e8895c86546 2023-03-22 15:55:27,097 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/7a1ee5b341654214b0149e8895c86546, entries=4, sequenceid=5, filesize=5.4 K 2023-03-22 15:55:27,097 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/d98c62e9217445fca22c34657d04df70 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/d98c62e9217445fca22c34657d04df70 2023-03-22 15:55:27,100 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/d98c62e9217445fca22c34657d04df70, entries=2, sequenceid=5, filesize=5.1 K 2023-03-22 15:55:27,101 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9ae79c6df98f35ced4ccb3fbc1d46164 in 1651ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:27,102 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:27,102 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:27,102 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:27,102 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/988ca6f3e4c946159ab0902697e2e4df 2023-03-22 15:55:27,105 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/2180a6d9f53a4db1ba6801549d1c6916 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2180a6d9f53a4db1ba6801549d1c6916 2023-03-22 15:55:27,108 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2180a6d9f53a4db1ba6801549d1c6916, entries=4, sequenceid=5, filesize=5.4 K 2023-03-22 15:55:27,109 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/988ca6f3e4c946159ab0902697e2e4df as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/988ca6f3e4c946159ab0902697e2e4df 2023-03-22 15:55:27,111 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/988ca6f3e4c946159ab0902697e2e4df, entries=2, sequenceid=5, filesize=5.1 K 2023-03-22 15:55:27,112 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 23b8dc9b2c9ae9fdf9b83a4c4302cb43 in 1663ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:27,112 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:27,112 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:27,112 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:27,117 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/b73492663d804f63b77ab70187f8e043 2023-03-22 15:55:27,120 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/30cb8b2d9a1a49a6af5e7e9b8c2cc2f2 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30cb8b2d9a1a49a6af5e7e9b8c2cc2f2 2023-03-22 15:55:27,123 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30cb8b2d9a1a49a6af5e7e9b8c2cc2f2, entries=4, sequenceid=5, filesize=5.4 K 2023-03-22 15:55:27,124 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/b73492663d804f63b77ab70187f8e043 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/b73492663d804f63b77ab70187f8e043 2023-03-22 15:55:27,126 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/b73492663d804f63b77ab70187f8e043, entries=2, sequenceid=5, filesize=5.1 K 2023-03-22 15:55:27,127 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 2c6ba5db98f0e0042380c0a57cebc9d0 in 1677ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:27,127 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:27,127 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:27,127 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:27,127 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:27,127 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,127 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-22 15:55:27,127 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-22 15:55:27,129 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:27,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-22 15:55:27,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2285664d[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:27,130 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-22 15:55:27,131 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:27,132 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:27,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-22 15:55:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-22 15:55:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:27,132 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,132 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:27,136 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,136 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-22 15:55:27,136 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:27,136 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-22 15:55:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:27,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-22 15:55:27,138 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-22 15:55:27,138 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:27,138 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:27,138 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-22 15:55:27,138 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:27,139 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:27,139 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:27,139 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:27,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:27,139 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:27,141 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,142 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:27,142 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:27,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:27,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-22 15:55:27,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:27,142 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:27,142 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-22 15:55:27,143 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:27,143 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:27,143 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:27,143 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:27,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:27,143 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,143 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:27,143 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:27,143 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:27,243 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:27,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:27,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:27,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:27,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:27,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:27,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:27,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-22 15:55:27,249 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-22 15:55:27,249 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:27,249 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-22 15:55:27,249 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-22 15:55:27,249 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:27,249 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:27,251 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,251 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:27,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:27,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:27,251 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:27,251 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:27,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:27,251 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:27,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:27,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:27,252 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-22 15:55:27,252 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-22 15:55:27,252 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-22 15:55:27,252 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-22 15:55:27,252 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:27,252 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-22 15:55:27,252 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:27,252 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. started... 2023-03-22 15:55:27,252 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:27,252 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:27,252 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:27,252 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. started... 2023-03-22 15:55:27,252 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. started... 2023-03-22 15:55:27,252 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ae79c6df98f35ced4ccb3fbc1d46164 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-22 15:55:27,252 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-22 15:55:27,252 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:27,252 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2c6ba5db98f0e0042380c0a57cebc9d0 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-22 15:55:27,665 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303223d539cbc4c3049ea8a1ab2065de6329e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303223d539cbc4c3049ea8a1ab2065de6329e 2023-03-22 15:55:27,665 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322a3c5ff7d3d9342c884a684a6edd9f335 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230322a3c5ff7d3d9342c884a684a6edd9f335 2023-03-22 15:55:27,665 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032245baffee42fc4f1ca5867434595583dd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023032245baffee42fc4f1ca5867434595583dd 2023-03-22 15:55:28,070 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/ff00cc72413c4c3f81ff2dee97a167b9 2023-03-22 15:55:28,070 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/30262f5953b04872be5ede6e37a5ef1c 2023-03-22 15:55:28,070 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/22315aa716124dac8572892d7bb33c13 2023-03-22 15:55:28,483 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303223786ca1725884cf9a71d585c636d5ec0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303223786ca1725884cf9a71d585c636d5ec0 2023-03-22 15:55:28,483 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322eb06810bb28b4e8ea573084df5917b62 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230322eb06810bb28b4e8ea573084df5917b62 2023-03-22 15:55:28,483 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303220fc6d7c5455f4364a68fecf34441427b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303220fc6d7c5455f4364a68fecf34441427b 2023-03-22 15:55:28,893 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/f1e86a1686ce4039985f508d9d498d6d 2023-03-22 15:55:28,893 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/5c357d5e551940b5abb388313a70cd6a 2023-03-22 15:55:28,893 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/030d2157ef604be6ac2b365e2c1b65f3 2023-03-22 15:55:28,897 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/22315aa716124dac8572892d7bb33c13 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/22315aa716124dac8572892d7bb33c13 2023-03-22 15:55:28,897 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/ff00cc72413c4c3f81ff2dee97a167b9 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/ff00cc72413c4c3f81ff2dee97a167b9 2023-03-22 15:55:28,897 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/30262f5953b04872be5ede6e37a5ef1c as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30262f5953b04872be5ede6e37a5ef1c 2023-03-22 15:55:28,901 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/ff00cc72413c4c3f81ff2dee97a167b9, entries=4, sequenceid=9, filesize=5.4 K 2023-03-22 15:55:28,901 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30262f5953b04872be5ede6e37a5ef1c, entries=4, sequenceid=9, filesize=5.4 K 2023-03-22 15:55:28,901 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/22315aa716124dac8572892d7bb33c13, entries=4, sequenceid=9, filesize=5.4 K 2023-03-22 15:55:28,901 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/5c357d5e551940b5abb388313a70cd6a as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/5c357d5e551940b5abb388313a70cd6a 2023-03-22 15:55:28,902 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/f1e86a1686ce4039985f508d9d498d6d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/f1e86a1686ce4039985f508d9d498d6d 2023-03-22 15:55:28,902 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/030d2157ef604be6ac2b365e2c1b65f3 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/030d2157ef604be6ac2b365e2c1b65f3 2023-03-22 15:55:28,906 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/f1e86a1686ce4039985f508d9d498d6d, entries=2, sequenceid=9, filesize=5.1 K 2023-03-22 15:55:28,906 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/5c357d5e551940b5abb388313a70cd6a, entries=2, sequenceid=9, filesize=5.1 K 2023-03-22 15:55:28,906 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/030d2157ef604be6ac2b365e2c1b65f3, entries=2, sequenceid=9, filesize=5.1 K 2023-03-22 15:55:28,906 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 2c6ba5db98f0e0042380c0a57cebc9d0 in 1654ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:28,907 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 23b8dc9b2c9ae9fdf9b83a4c4302cb43 in 1654ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:28,907 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9ae79c6df98f35ced4ccb3fbc1d46164 in 1654ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:28,907 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:28,907 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:28,907 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:28,907 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:28,907 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:28,907 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:28,907 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:28,907 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:28,907 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:28,907 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:28,907 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,907 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-22 15:55:28,907 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-22 15:55:28,910 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,910 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:28,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:28,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:28,910 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:28,910 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:28,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:28,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-22 15:55:28,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f227d24[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:28,912 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-22 15:55:28,912 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:28,913 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:28,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:28,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:28,913 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-22 15:55:28,913 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-22 15:55:28,913 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:28,913 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,913 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:28,918 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-22 15:55:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:28,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:28,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:28,918 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-22 15:55:28,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:28,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:28,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:28,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:28,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:28,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:28,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-22 15:55:28,920 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-22 15:55:28,921 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:28,921 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:28,921 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-22 15:55:28,921 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:28,922 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:28,922 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:28,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:28,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:28,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:28,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:28,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:28,922 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:28,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:28,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:28,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:28,923 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:28,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:28,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:28,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:28,924 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:28,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:28,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:28,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:28,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:28,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:28,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:28,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:28,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-22 15:55:28,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:28,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-22 15:55:28,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:28,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:28,926 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:28,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:28,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:28,927 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:28,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:28,927 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:29,027 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:29,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:29,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:29,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:29,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:29,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:29,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:29,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-22 15:55:29,033 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-22 15:55:29,033 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:29,033 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-22 15:55:29,033 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-22 15:55:29,034 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:29,034 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:29,036 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:29,036 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:29,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:29,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:29,036 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:29,036 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:29,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:29,037 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:29,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:29,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:29,037 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-22 15:55:29,037 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-22 15:55:29,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-22 15:55:29,037 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-22 15:55:29,037 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:29,038 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-22 15:55:29,043 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:29,043 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:29,043 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:29,043 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:29,043 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. started... 2023-03-22 15:55:29,043 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. started... 2023-03-22 15:55:29,043 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:29,043 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. started... 2023-03-22 15:55:29,043 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-22 15:55:29,043 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2c6ba5db98f0e0042380c0a57cebc9d0 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-22 15:55:29,043 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ae79c6df98f35ced4ccb3fbc1d46164 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-22 15:55:29,057 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322f763f9c8c26c4fa2b139870e9d9f85de to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230322f763f9c8c26c4fa2b139870e9d9f85de 2023-03-22 15:55:29,063 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/d0b9d1bbc4c74863a734eacda996e483 2023-03-22 15:55:29,455 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303228ca059b0a8354a5ab2ecf385c1de5036 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303228ca059b0a8354a5ab2ecf385c1de5036 2023-03-22 15:55:29,457 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032209ee89e9b9864ff3852e9b94946b4566 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023032209ee89e9b9864ff3852e9b94946b4566 2023-03-22 15:55:29,479 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303220d4fbcbc99a0474a9a73352d816810eb to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303220d4fbcbc99a0474a9a73352d816810eb 2023-03-22 15:55:29,860 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/806abca956cc4e0aafef82d6bf3b5f18 2023-03-22 15:55:29,861 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/b43e6d5b0c004bdd81555a4a6d235d5c 2023-03-22 15:55:29,882 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/a6897edbdf5f4082bc08c40c9656705a 2023-03-22 15:55:29,886 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/d0b9d1bbc4c74863a734eacda996e483 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/d0b9d1bbc4c74863a734eacda996e483 2023-03-22 15:55:29,889 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/d0b9d1bbc4c74863a734eacda996e483, entries=4, sequenceid=13, filesize=5.4 K 2023-03-22 15:55:29,889 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/a6897edbdf5f4082bc08c40c9656705a as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a6897edbdf5f4082bc08c40c9656705a 2023-03-22 15:55:29,892 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a6897edbdf5f4082bc08c40c9656705a, entries=2, sequenceid=13, filesize=5.1 K 2023-03-22 15:55:29,892 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 2c6ba5db98f0e0042380c0a57cebc9d0 in 849ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:29,893 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:29,893 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:29,893 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:29,995 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:55:30,273 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303229e029c4fe87945298fa0b7e5941b07dd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303229e029c4fe87945298fa0b7e5941b07dd 2023-03-22 15:55:30,273 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303224f75f7391f7f4d439c0afb1c7e5001e4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303224f75f7391f7f4d439c0afb1c7e5001e4 2023-03-22 15:55:30,283 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/57f909e7412c4a25a2fd9ff1ac9dcd2b 2023-03-22 15:55:30,286 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/806abca956cc4e0aafef82d6bf3b5f18 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/806abca956cc4e0aafef82d6bf3b5f18 2023-03-22 15:55:30,289 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/806abca956cc4e0aafef82d6bf3b5f18, entries=4, sequenceid=13, filesize=5.4 K 2023-03-22 15:55:30,290 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/57f909e7412c4a25a2fd9ff1ac9dcd2b as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/57f909e7412c4a25a2fd9ff1ac9dcd2b 2023-03-22 15:55:30,292 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/57f909e7412c4a25a2fd9ff1ac9dcd2b, entries=2, sequenceid=13, filesize=5.1 K 2023-03-22 15:55:30,293 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 23b8dc9b2c9ae9fdf9b83a4c4302cb43 in 1250ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:30,293 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:30,293 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:30,293 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:30,680 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/9126f14ca2124992a4535c62045d7bc8 2023-03-22 15:55:30,684 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/b43e6d5b0c004bdd81555a4a6d235d5c as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/b43e6d5b0c004bdd81555a4a6d235d5c 2023-03-22 15:55:30,687 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/b43e6d5b0c004bdd81555a4a6d235d5c, entries=4, sequenceid=13, filesize=5.4 K 2023-03-22 15:55:30,687 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/9126f14ca2124992a4535c62045d7bc8 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/9126f14ca2124992a4535c62045d7bc8 2023-03-22 15:55:30,690 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/9126f14ca2124992a4535c62045d7bc8, entries=2, sequenceid=13, filesize=5.1 K 2023-03-22 15:55:30,691 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9ae79c6df98f35ced4ccb3fbc1d46164 in 1648ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:30,691 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:30,691 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:30,691 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:30,691 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:30,691 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,691 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-22 15:55:30,691 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-22 15:55:30,693 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:30,693 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:30,693 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:30,693 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:30,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:30,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:30,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-22 15:55:30,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d4a7a8f[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:30,694 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-22 15:55:30,694 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:30,695 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:30,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:30,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:30,696 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-22 15:55:30,696 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-22 15:55:30,696 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,696 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:30,696 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:30,697 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-22 15:55:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-22 15:55:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:30,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-22 15:55:30,699 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-22 15:55:30,699 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:30,699 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:30,699 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-22 15:55:30,699 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:30,701 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:30,701 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:30,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:30,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:30,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:30,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:30,701 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:30,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:30,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:30,701 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:30,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:30,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:30,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:30,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:30,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:30,702 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:30,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:30,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:30,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:30,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-22 15:55:30,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:30,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:30,705 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:30,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:30,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:30,705 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:30,805 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:30,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:30,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:30,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:30,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:30,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:30,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:30,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-22 15:55:30,811 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-22 15:55:30,811 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:30,811 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-22 15:55:30,811 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-22 15:55:30,812 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:30,812 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:30,813 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,813 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:30,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:30,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:30,814 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:30,814 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:30,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:30,814 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-22 15:55:30,814 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-22 15:55:30,814 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-22 15:55:30,815 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-22 15:55:30,815 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:30,815 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-22 15:55:30,816 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:30,816 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. started... 2023-03-22 15:55:30,816 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:30,816 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:30,816 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:30,816 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:30,817 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. started... 2023-03-22 15:55:30,816 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ae79c6df98f35ced4ccb3fbc1d46164 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-22 15:55:30,817 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-22 15:55:30,816 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. started... 2023-03-22 15:55:30,817 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2c6ba5db98f0e0042380c0a57cebc9d0 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-22 15:55:30,836 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303223befd81bd87843c1aa698d88a0aac812 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303223befd81bd87843c1aa698d88a0aac812 2023-03-22 15:55:30,837 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032246f39909aac14ffbb74020a36d2b1073 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023032246f39909aac14ffbb74020a36d2b1073 2023-03-22 15:55:30,840 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/5bb7544179f940c1a784b1419ac6e256 2023-03-22 15:55:30,850 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322449c525a8c67496a9cf79d18b91112d1 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230322449c525a8c67496a9cf79d18b91112d1 2023-03-22 15:55:31,235 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322b25cd6c90b0541d992c380b591618b45 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230322b25cd6c90b0541d992c380b591618b45 2023-03-22 15:55:31,241 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/cd7836faf5654b27b2ed06e8ae142283 2023-03-22 15:55:31,254 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/831b9d914ea94e53ae7cad5158a98436 2023-03-22 15:55:31,257 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/5bb7544179f940c1a784b1419ac6e256 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/5bb7544179f940c1a784b1419ac6e256 2023-03-22 15:55:31,260 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/5bb7544179f940c1a784b1419ac6e256, entries=4, sequenceid=17, filesize=5.4 K 2023-03-22 15:55:31,261 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/831b9d914ea94e53ae7cad5158a98436 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/831b9d914ea94e53ae7cad5158a98436 2023-03-22 15:55:31,263 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/831b9d914ea94e53ae7cad5158a98436, entries=2, sequenceid=17, filesize=5.1 K 2023-03-22 15:55:31,264 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 2c6ba5db98f0e0042380c0a57cebc9d0 in 447ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:31,264 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:31,264 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:31,264 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:31,640 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/f34eac41d23a43518532db750ac6f48f 2023-03-22 15:55:31,652 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322f12559114e3f49bc99eff06269357c2e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230322f12559114e3f49bc99eff06269357c2e 2023-03-22 15:55:32,053 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322648dc3d336ed4b94b525b6f7552945c2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230322648dc3d336ed4b94b525b6f7552945c2 2023-03-22 15:55:32,056 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/1a32a166d2a44edf93d19f8dd5b267da 2023-03-22 15:55:32,059 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/cd7836faf5654b27b2ed06e8ae142283 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/cd7836faf5654b27b2ed06e8ae142283 2023-03-22 15:55:32,062 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/cd7836faf5654b27b2ed06e8ae142283, entries=4, sequenceid=17, filesize=5.4 K 2023-03-22 15:55:32,062 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/1a32a166d2a44edf93d19f8dd5b267da as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/1a32a166d2a44edf93d19f8dd5b267da 2023-03-22 15:55:32,065 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/1a32a166d2a44edf93d19f8dd5b267da, entries=2, sequenceid=17, filesize=5.1 K 2023-03-22 15:55:32,066 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9ae79c6df98f35ced4ccb3fbc1d46164 in 1250ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:32,066 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:32,066 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:32,066 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:32,457 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/aac4a960c7424ad2bb30795a4f3c4a88 2023-03-22 15:55:32,461 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/f34eac41d23a43518532db750ac6f48f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/f34eac41d23a43518532db750ac6f48f 2023-03-22 15:55:32,464 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/f34eac41d23a43518532db750ac6f48f, entries=4, sequenceid=17, filesize=5.4 K 2023-03-22 15:55:32,465 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/aac4a960c7424ad2bb30795a4f3c4a88 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/aac4a960c7424ad2bb30795a4f3c4a88 2023-03-22 15:55:32,468 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/aac4a960c7424ad2bb30795a4f3c4a88, entries=2, sequenceid=17, filesize=5.1 K 2023-03-22 15:55:32,469 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 23b8dc9b2c9ae9fdf9b83a4c4302cb43 in 1652ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:32,469 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:32,469 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:32,469 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:32,469 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:32,469 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,469 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-22 15:55:32,469 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-22 15:55:32,470 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:32,470 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:32,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:32,471 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:32,471 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:32,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:32,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:32,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:32,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:32,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-22 15:55:32,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41a6d25a[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:32,472 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-22 15:55:32,472 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:32,474 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:32,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:32,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:32,474 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-22 15:55:32,474 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,474 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-22 15:55:32,474 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:32,474 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:32,475 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-22 15:55:32,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:32,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:32,476 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-22 15:55:32,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:32,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:32,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:32,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:32,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:32,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:32,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-22 15:55:32,477 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-22 15:55:32,477 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:32,477 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:32,477 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-22 15:55:32,477 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:32,479 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:32,479 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:32,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:32,479 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:32,479 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:32,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:32,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:32,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:32,480 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:32,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:32,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,482 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,482 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:32,482 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:32,482 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:32,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:32,482 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-22 15:55:32,482 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:32,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-22 15:55:32,483 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:32,482 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:32,483 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:32,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:32,483 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:32,483 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:32,483 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:32,483 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:32,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:32,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:32,583 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:32,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:32,591 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:32,596 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:32,603 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:32,604 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:32,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:32,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:32,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-22 15:55:32,631 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-22 15:55:32,631 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:32,631 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-22 15:55:32,631 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-22 15:55:32,631 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:32,631 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:32,635 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,635 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:32,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:32,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:32,635 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:32,635 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:32,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:32,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:32,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:32,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:32,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-22 15:55:32,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-22 15:55:32,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-22 15:55:32,636 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-22 15:55:32,636 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:32,636 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-22 15:55:32,636 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:32,636 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:32,636 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:32,636 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:32,636 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:32,637 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. started... 2023-03-22 15:55:32,636 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. started... 2023-03-22 15:55:32,636 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. started... 2023-03-22 15:55:32,637 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-22 15:55:32,637 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2c6ba5db98f0e0042380c0a57cebc9d0 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-22 15:55:32,637 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ae79c6df98f35ced4ccb3fbc1d46164 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-22 15:55:33,050 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/2cfebf8202f34c90b4232b76b76c7687 2023-03-22 15:55:33,051 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/ba3f407262504d7ba70619ee82b3a12f 2023-03-22 15:55:33,054 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2cfebf8202f34c90b4232b76b76c7687 2023-03-22 15:55:33,054 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ba3f407262504d7ba70619ee82b3a12f 2023-03-22 15:55:33,449 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/a62e4cf64c2244b190af09f38f85aeea 2023-03-22 15:55:33,453 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a62e4cf64c2244b190af09f38f85aeea 2023-03-22 15:55:33,465 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/d294e09e235a428cad90c948e507ad01 2023-03-22 15:55:33,468 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d294e09e235a428cad90c948e507ad01 2023-03-22 15:55:33,469 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/2cfebf8202f34c90b4232b76b76c7687 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2cfebf8202f34c90b4232b76b76c7687 2023-03-22 15:55:33,471 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2cfebf8202f34c90b4232b76b76c7687 2023-03-22 15:55:33,472 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2cfebf8202f34c90b4232b76b76c7687, entries=3, sequenceid=23, filesize=5.0 K 2023-03-22 15:55:33,472 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/d294e09e235a428cad90c948e507ad01 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/d294e09e235a428cad90c948e507ad01 2023-03-22 15:55:33,475 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d294e09e235a428cad90c948e507ad01 2023-03-22 15:55:33,475 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/d294e09e235a428cad90c948e507ad01, entries=1, sequenceid=23, filesize=5.0 K 2023-03-22 15:55:33,475 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 23b8dc9b2c9ae9fdf9b83a4c4302cb43 in 838ms, sequenceid=23, compaction requested=true 2023-03-22 15:55:33,476 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:33,476 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:33,476 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:33,865 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/a1606941f1e6490c808f683b5e5c13f3 2023-03-22 15:55:33,868 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a1606941f1e6490c808f683b5e5c13f3 2023-03-22 15:55:33,869 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/ba3f407262504d7ba70619ee82b3a12f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/ba3f407262504d7ba70619ee82b3a12f 2023-03-22 15:55:33,872 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ba3f407262504d7ba70619ee82b3a12f 2023-03-22 15:55:33,872 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/ba3f407262504d7ba70619ee82b3a12f, entries=3, sequenceid=23, filesize=5.0 K 2023-03-22 15:55:33,873 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/a1606941f1e6490c808f683b5e5c13f3 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a1606941f1e6490c808f683b5e5c13f3 2023-03-22 15:55:33,876 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a1606941f1e6490c808f683b5e5c13f3 2023-03-22 15:55:33,876 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a1606941f1e6490c808f683b5e5c13f3, entries=1, sequenceid=23, filesize=5.0 K 2023-03-22 15:55:33,877 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 2c6ba5db98f0e0042380c0a57cebc9d0 in 1240ms, sequenceid=23, compaction requested=true 2023-03-22 15:55:33,877 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:33,877 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:33,877 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:34,263 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/72db2eac45a74bad8cd5ac44ac49a3c7 2023-03-22 15:55:34,266 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72db2eac45a74bad8cd5ac44ac49a3c7 2023-03-22 15:55:34,267 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/a62e4cf64c2244b190af09f38f85aeea as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/a62e4cf64c2244b190af09f38f85aeea 2023-03-22 15:55:34,270 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a62e4cf64c2244b190af09f38f85aeea 2023-03-22 15:55:34,270 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/a62e4cf64c2244b190af09f38f85aeea, entries=3, sequenceid=23, filesize=5.0 K 2023-03-22 15:55:34,270 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/72db2eac45a74bad8cd5ac44ac49a3c7 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/72db2eac45a74bad8cd5ac44ac49a3c7 2023-03-22 15:55:34,273 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72db2eac45a74bad8cd5ac44ac49a3c7 2023-03-22 15:55:34,273 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/72db2eac45a74bad8cd5ac44ac49a3c7, entries=1, sequenceid=23, filesize=5.0 K 2023-03-22 15:55:34,274 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9ae79c6df98f35ced4ccb3fbc1d46164 in 1637ms, sequenceid=23, compaction requested=true 2023-03-22 15:55:34,274 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:34,274 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:34,274 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:34,274 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:34,274 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,274 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-22 15:55:34,274 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-22 15:55:34,276 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:34,276 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:34,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:34,276 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:34,276 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:34,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:34,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:34,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:34,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:34,277 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-22 15:55:34,277 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4818632d[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:34,277 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-22 15:55:34,277 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:34,278 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:34,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:34,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:34,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-22 15:55:34,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-22 15:55:34,279 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:34,279 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,279 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:34,280 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-22 15:55:34,280 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,280 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:34,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:34,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:34,280 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-22 15:55:34,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:34,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:34,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:34,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:34,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:34,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:34,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-22 15:55:34,282 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-22 15:55:34,282 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:34,282 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:34,282 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-22 15:55:34,282 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:34,287 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:34,287 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:34,287 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:34,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:34,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:34,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:34,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:34,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:34,287 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:34,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:34,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:34,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:34,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:34,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:34,288 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:34,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:34,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:34,290 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:34,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-22 15:55:34,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:34,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:34,290 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:34,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:34,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-22 15:55:34,290 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:34,391 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:34,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:34,392 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:34,392 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 9ae79c6df98f35ced4ccb3fbc1d46164/family1 is initiating major compaction (all files) 2023-03-22 15:55:34,392 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:34,392 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:34,392 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 9ae79c6df98f35ced4ccb3fbc1d46164/family1 in ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:34,392 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/7a1ee5b341654214b0149e8895c86546, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/ff00cc72413c4c3f81ff2dee97a167b9, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/b43e6d5b0c004bdd81555a4a6d235d5c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/cd7836faf5654b27b2ed06e8ae142283, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/a62e4cf64c2244b190af09f38f85aeea] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp, totalSize=26.6 K 2023-03-22 15:55:34,393 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 7a1ee5b341654214b0149e8895c86546, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500525440 2023-03-22 15:55:34,393 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting ff00cc72413c4c3f81ff2dee97a167b9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500527245 2023-03-22 15:55:34,394 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting b43e6d5b0c004bdd81555a4a6d235d5c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500529029 2023-03-22 15:55:34,394 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting cd7836faf5654b27b2ed06e8ae142283, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500530807 2023-03-22 15:55:34,394 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting a62e4cf64c2244b190af09f38f85aeea, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:55:34,403 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 9ae79c6df98f35ced4ccb3fbc1d46164#family1#compaction#150 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:34,812 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230322f75c46ac95c849aea4dec8856aff7420_del 2023-03-22 15:55:34,812 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322f75c46ac95c849aea4dec8856aff7420_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230322f75c46ac95c849aea4dec8856aff7420_del 2023-03-22 15:55:35,624 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2910d90475644abfba5ae9baf7dee44e 2023-03-22 15:55:35,625 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family1/2910d90475644abfba5ae9baf7dee44e as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/2910d90475644abfba5ae9baf7dee44e 2023-03-22 15:55:35,628 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2910d90475644abfba5ae9baf7dee44e 2023-03-22 15:55:35,629 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9ae79c6df98f35ced4ccb3fbc1d46164/family1 of 9ae79c6df98f35ced4ccb3fbc1d46164 into 2910d90475644abfba5ae9baf7dee44e(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:35,629 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:35,629 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:35,629 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 9ae79c6df98f35ced4ccb3fbc1d46164/family2 is initiating major compaction (all files) 2023-03-22 15:55:35,629 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:35,629 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:35,629 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 9ae79c6df98f35ced4ccb3fbc1d46164/family2 in ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:55:35,629 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/d98c62e9217445fca22c34657d04df70, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/5c357d5e551940b5abb388313a70cd6a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/9126f14ca2124992a4535c62045d7bc8, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/1a32a166d2a44edf93d19f8dd5b267da, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/72db2eac45a74bad8cd5ac44ac49a3c7] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp, totalSize=25.3 K 2023-03-22 15:55:35,630 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting d98c62e9217445fca22c34657d04df70, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500525440 2023-03-22 15:55:35,630 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 5c357d5e551940b5abb388313a70cd6a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500527245 2023-03-22 15:55:35,630 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 9126f14ca2124992a4535c62045d7bc8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500529029 2023-03-22 15:55:35,631 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 1a32a166d2a44edf93d19f8dd5b267da, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500530807 2023-03-22 15:55:35,631 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 72db2eac45a74bad8cd5ac44ac49a3c7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:55:35,638 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 9ae79c6df98f35ced4ccb3fbc1d46164#family2#compaction#151 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:35,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/7a1ee5b341654214b0149e8895c86546, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/ff00cc72413c4c3f81ff2dee97a167b9, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/b43e6d5b0c004bdd81555a4a6d235d5c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/cd7836faf5654b27b2ed06e8ae142283, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/a62e4cf64c2244b190af09f38f85aeea] to archive 2023-03-22 15:55:35,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:35,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/7a1ee5b341654214b0149e8895c86546 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/7a1ee5b341654214b0149e8895c86546 2023-03-22 15:55:35,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/ff00cc72413c4c3f81ff2dee97a167b9 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/ff00cc72413c4c3f81ff2dee97a167b9 2023-03-22 15:55:35,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/b43e6d5b0c004bdd81555a4a6d235d5c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/b43e6d5b0c004bdd81555a4a6d235d5c 2023-03-22 15:55:35,716 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/cd7836faf5654b27b2ed06e8ae142283 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/cd7836faf5654b27b2ed06e8ae142283 2023-03-22 15:55:35,717 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/a62e4cf64c2244b190af09f38f85aeea to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/a62e4cf64c2244b190af09f38f85aeea 2023-03-22 15:55:36,046 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230322e4ec9b50a66c4f15a0f30b331a1e68d2_del 2023-03-22 15:55:36,046 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322e4ec9b50a66c4f15a0f30b331a1e68d2_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230322e4ec9b50a66c4f15a0f30b331a1e68d2_del 2023-03-22 15:55:36,859 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f76ecb0766464cc2b7ac1b7c54a84fb3 2023-03-22 15:55:36,860 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/.tmp/family2/f76ecb0766464cc2b7ac1b7c54a84fb3 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/f76ecb0766464cc2b7ac1b7c54a84fb3 2023-03-22 15:55:36,863 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f76ecb0766464cc2b7ac1b7c54a84fb3 2023-03-22 15:55:36,864 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9ae79c6df98f35ced4ccb3fbc1d46164/family2 of 9ae79c6df98f35ced4ccb3fbc1d46164 into f76ecb0766464cc2b7ac1b7c54a84fb3(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:36,864 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:36,865 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:36,865 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1 is initiating major compaction (all files) 2023-03-22 15:55:36,865 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:36,865 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:36,865 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1 in ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:36,865 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2180a6d9f53a4db1ba6801549d1c6916, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/22315aa716124dac8572892d7bb33c13, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/806abca956cc4e0aafef82d6bf3b5f18, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/f34eac41d23a43518532db750ac6f48f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2cfebf8202f34c90b4232b76b76c7687] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp, totalSize=26.6 K 2023-03-22 15:55:36,865 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 2180a6d9f53a4db1ba6801549d1c6916, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500525441 2023-03-22 15:55:36,866 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 22315aa716124dac8572892d7bb33c13, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500527245 2023-03-22 15:55:36,866 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 806abca956cc4e0aafef82d6bf3b5f18, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500529029 2023-03-22 15:55:36,867 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting f34eac41d23a43518532db750ac6f48f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500530807 2023-03-22 15:55:36,867 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 2cfebf8202f34c90b4232b76b76c7687, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:55:36,876 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 23b8dc9b2c9ae9fdf9b83a4c4302cb43#family1#compaction#152 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:36,910 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/d98c62e9217445fca22c34657d04df70, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/5c357d5e551940b5abb388313a70cd6a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/9126f14ca2124992a4535c62045d7bc8, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/1a32a166d2a44edf93d19f8dd5b267da, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/72db2eac45a74bad8cd5ac44ac49a3c7] to archive 2023-03-22 15:55:36,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:36,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/d98c62e9217445fca22c34657d04df70 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/d98c62e9217445fca22c34657d04df70 2023-03-22 15:55:36,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/5c357d5e551940b5abb388313a70cd6a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/5c357d5e551940b5abb388313a70cd6a 2023-03-22 15:55:36,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/9126f14ca2124992a4535c62045d7bc8 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/9126f14ca2124992a4535c62045d7bc8 2023-03-22 15:55:36,916 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/1a32a166d2a44edf93d19f8dd5b267da to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/1a32a166d2a44edf93d19f8dd5b267da 2023-03-22 15:55:36,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/72db2eac45a74bad8cd5ac44ac49a3c7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family2/72db2eac45a74bad8cd5ac44ac49a3c7 2023-03-22 15:55:37,283 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303224164bd465e864ca091fe4590f9ce4f84_del 2023-03-22 15:55:37,283 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303224164bd465e864ca091fe4590f9ce4f84_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303224164bd465e864ca091fe4590f9ce4f84_del 2023-03-22 15:55:38,096 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 872cf9e395ba47f3bc5625321164a29a 2023-03-22 15:55:38,096 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family1/872cf9e395ba47f3bc5625321164a29a as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/872cf9e395ba47f3bc5625321164a29a 2023-03-22 15:55:38,100 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 872cf9e395ba47f3bc5625321164a29a 2023-03-22 15:55:38,101 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1 of 23b8dc9b2c9ae9fdf9b83a4c4302cb43 into 872cf9e395ba47f3bc5625321164a29a(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:38,101 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:38,101 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:38,101 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2 is initiating major compaction (all files) 2023-03-22 15:55:38,101 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:38,101 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:38,102 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2 in ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:55:38,102 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/988ca6f3e4c946159ab0902697e2e4df, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/030d2157ef604be6ac2b365e2c1b65f3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/57f909e7412c4a25a2fd9ff1ac9dcd2b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/aac4a960c7424ad2bb30795a4f3c4a88, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/d294e09e235a428cad90c948e507ad01] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp, totalSize=25.3 K 2023-03-22 15:55:38,102 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 988ca6f3e4c946159ab0902697e2e4df, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500525441 2023-03-22 15:55:38,102 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 030d2157ef604be6ac2b365e2c1b65f3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500527245 2023-03-22 15:55:38,103 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 57f909e7412c4a25a2fd9ff1ac9dcd2b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500529029 2023-03-22 15:55:38,103 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting aac4a960c7424ad2bb30795a4f3c4a88, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500530807 2023-03-22 15:55:38,103 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting d294e09e235a428cad90c948e507ad01, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:55:38,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2180a6d9f53a4db1ba6801549d1c6916, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/22315aa716124dac8572892d7bb33c13, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/806abca956cc4e0aafef82d6bf3b5f18, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/f34eac41d23a43518532db750ac6f48f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2cfebf8202f34c90b4232b76b76c7687] to archive 2023-03-22 15:55:38,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:38,112 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 23b8dc9b2c9ae9fdf9b83a4c4302cb43#family2#compaction#153 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:38,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2180a6d9f53a4db1ba6801549d1c6916 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2180a6d9f53a4db1ba6801549d1c6916 2023-03-22 15:55:38,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/22315aa716124dac8572892d7bb33c13 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/22315aa716124dac8572892d7bb33c13 2023-03-22 15:55:38,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/806abca956cc4e0aafef82d6bf3b5f18 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/806abca956cc4e0aafef82d6bf3b5f18 2023-03-22 15:55:38,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/f34eac41d23a43518532db750ac6f48f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/f34eac41d23a43518532db750ac6f48f 2023-03-22 15:55:38,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2cfebf8202f34c90b4232b76b76c7687 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/2cfebf8202f34c90b4232b76b76c7687 2023-03-22 15:55:38,118 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032298416dd9cb5f4b6d945995e058325554_del 2023-03-22 15:55:38,118 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032298416dd9cb5f4b6d945995e058325554_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023032298416dd9cb5f4b6d945995e058325554_del 2023-03-22 15:55:38,931 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8464438cbb3a4b919d5fcf4562a096f7 2023-03-22 15:55:38,932 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/.tmp/family2/8464438cbb3a4b919d5fcf4562a096f7 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/8464438cbb3a4b919d5fcf4562a096f7 2023-03-22 15:55:38,935 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8464438cbb3a4b919d5fcf4562a096f7 2023-03-22 15:55:38,936 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2 of 23b8dc9b2c9ae9fdf9b83a4c4302cb43 into 8464438cbb3a4b919d5fcf4562a096f7(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:38,936 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:38,936 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:38,936 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 2c6ba5db98f0e0042380c0a57cebc9d0/family1 is initiating major compaction (all files) 2023-03-22 15:55:38,936 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:38,936 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:38,936 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 2c6ba5db98f0e0042380c0a57cebc9d0/family1 in ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:38,936 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30cb8b2d9a1a49a6af5e7e9b8c2cc2f2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30262f5953b04872be5ede6e37a5ef1c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/d0b9d1bbc4c74863a734eacda996e483, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/5bb7544179f940c1a784b1419ac6e256, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/ba3f407262504d7ba70619ee82b3a12f] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp, totalSize=26.6 K 2023-03-22 15:55:38,937 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 30cb8b2d9a1a49a6af5e7e9b8c2cc2f2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500525441 2023-03-22 15:55:38,937 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 30262f5953b04872be5ede6e37a5ef1c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500527245 2023-03-22 15:55:38,937 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting d0b9d1bbc4c74863a734eacda996e483, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500529029 2023-03-22 15:55:38,937 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 5bb7544179f940c1a784b1419ac6e256, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500530807 2023-03-22 15:55:38,938 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting ba3f407262504d7ba70619ee82b3a12f, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:55:38,945 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 2c6ba5db98f0e0042380c0a57cebc9d0#family1#compaction#154 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:39,012 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/988ca6f3e4c946159ab0902697e2e4df, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/030d2157ef604be6ac2b365e2c1b65f3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/57f909e7412c4a25a2fd9ff1ac9dcd2b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/aac4a960c7424ad2bb30795a4f3c4a88, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/d294e09e235a428cad90c948e507ad01] to archive 2023-03-22 15:55:39,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:39,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/988ca6f3e4c946159ab0902697e2e4df to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/988ca6f3e4c946159ab0902697e2e4df 2023-03-22 15:55:39,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/030d2157ef604be6ac2b365e2c1b65f3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/030d2157ef604be6ac2b365e2c1b65f3 2023-03-22 15:55:39,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/57f909e7412c4a25a2fd9ff1ac9dcd2b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/57f909e7412c4a25a2fd9ff1ac9dcd2b 2023-03-22 15:55:39,017 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/aac4a960c7424ad2bb30795a4f3c4a88 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/aac4a960c7424ad2bb30795a4f3c4a88 2023-03-22 15:55:39,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/d294e09e235a428cad90c948e507ad01 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family2/d294e09e235a428cad90c948e507ad01 2023-03-22 15:55:39,352 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023032296091051de7c4bb784590467bb990bbd_del 2023-03-22 15:55:39,352 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032296091051de7c4bb784590467bb990bbd_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023032296091051de7c4bb784590467bb990bbd_del 2023-03-22 15:55:40,173 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ad5a9e8b5bc43c6bd6ccee1cd29af94 2023-03-22 15:55:40,174 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family1/1ad5a9e8b5bc43c6bd6ccee1cd29af94 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/1ad5a9e8b5bc43c6bd6ccee1cd29af94 2023-03-22 15:55:40,182 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ad5a9e8b5bc43c6bd6ccee1cd29af94 2023-03-22 15:55:40,184 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2c6ba5db98f0e0042380c0a57cebc9d0/family1 of 2c6ba5db98f0e0042380c0a57cebc9d0 into 1ad5a9e8b5bc43c6bd6ccee1cd29af94(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:40,184 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:40,184 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:40,184 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 2c6ba5db98f0e0042380c0a57cebc9d0/family2 is initiating major compaction (all files) 2023-03-22 15:55:40,184 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:40,184 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:40,184 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 2c6ba5db98f0e0042380c0a57cebc9d0/family2 in ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:55:40,185 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/b73492663d804f63b77ab70187f8e043, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/f1e86a1686ce4039985f508d9d498d6d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a6897edbdf5f4082bc08c40c9656705a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/831b9d914ea94e53ae7cad5158a98436, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a1606941f1e6490c808f683b5e5c13f3] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp, totalSize=25.3 K 2023-03-22 15:55:40,185 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting b73492663d804f63b77ab70187f8e043, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500525441 2023-03-22 15:55:40,186 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting f1e86a1686ce4039985f508d9d498d6d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500527245 2023-03-22 15:55:40,187 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting a6897edbdf5f4082bc08c40c9656705a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500529029 2023-03-22 15:55:40,187 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 831b9d914ea94e53ae7cad5158a98436, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500530807 2023-03-22 15:55:40,187 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting a1606941f1e6490c808f683b5e5c13f3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:55:40,203 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 2c6ba5db98f0e0042380c0a57cebc9d0#family2#compaction#155 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:40,212 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30cb8b2d9a1a49a6af5e7e9b8c2cc2f2, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30262f5953b04872be5ede6e37a5ef1c, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/d0b9d1bbc4c74863a734eacda996e483, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/5bb7544179f940c1a784b1419ac6e256, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/ba3f407262504d7ba70619ee82b3a12f] to archive 2023-03-22 15:55:40,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:40,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30cb8b2d9a1a49a6af5e7e9b8c2cc2f2 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30cb8b2d9a1a49a6af5e7e9b8c2cc2f2 2023-03-22 15:55:40,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30262f5953b04872be5ede6e37a5ef1c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/30262f5953b04872be5ede6e37a5ef1c 2023-03-22 15:55:40,219 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/d0b9d1bbc4c74863a734eacda996e483 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/d0b9d1bbc4c74863a734eacda996e483 2023-03-22 15:55:40,220 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/5bb7544179f940c1a784b1419ac6e256 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/5bb7544179f940c1a784b1419ac6e256 2023-03-22 15:55:40,222 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/ba3f407262504d7ba70619ee82b3a12f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/ba3f407262504d7ba70619ee82b3a12f 2023-03-22 15:55:40,230 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230322350ad14aac0c421aaff4cac8b6af653d_del 2023-03-22 15:55:40,230 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322350ad14aac0c421aaff4cac8b6af653d_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230322350ad14aac0c421aaff4cac8b6af653d_del 2023-03-22 15:55:41,045 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e294b6f9b64e4ed6b373f6c5f6f877bd 2023-03-22 15:55:41,046 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/.tmp/family2/e294b6f9b64e4ed6b373f6c5f6f877bd as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/e294b6f9b64e4ed6b373f6c5f6f877bd 2023-03-22 15:55:41,049 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e294b6f9b64e4ed6b373f6c5f6f877bd 2023-03-22 15:55:41,050 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2c6ba5db98f0e0042380c0a57cebc9d0/family2 of 2c6ba5db98f0e0042380c0a57cebc9d0 into e294b6f9b64e4ed6b373f6c5f6f877bd(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:41,050 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:41,054 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:41,057 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:41,061 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:41,063 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:41,069 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-22 15:55:41,083 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-22 15:55:41,086 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230322f75c46ac95c849aea4dec8856aff7420_del 2023-03-22 15:55:41,090 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303224164bd465e864ca091fe4590f9ce4f84_del 2023-03-22 15:55:41,094 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023032296091051de7c4bb784590467bb990bbd_del 2023-03-22 15:55:41,094 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-22 15:55:41,094 DEBUG [Listener at localhost/43289] zookeeper.ReadOnlyZKClient(139): Connect 0x4ee95ad1 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:55:41,101 DEBUG [Listener at localhost/43289] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@44840766, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:55:41,102 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230322 for table='ns:testMinorCompaction' and column='family1' 2023-03-22 15:55:41,102 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230322 for table='ns:testMinorCompaction' and column='family1' 2023-03-22 15:55:41,102 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230322 for table='ns:testMinorCompaction' and column='family1' 2023-03-22 15:55:41,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/b73492663d804f63b77ab70187f8e043, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/f1e86a1686ce4039985f508d9d498d6d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a6897edbdf5f4082bc08c40c9656705a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/831b9d914ea94e53ae7cad5158a98436, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a1606941f1e6490c808f683b5e5c13f3] to archive 2023-03-22 15:55:41,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:41,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/b73492663d804f63b77ab70187f8e043 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/b73492663d804f63b77ab70187f8e043 2023-03-22 15:55:41,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/f1e86a1686ce4039985f508d9d498d6d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/f1e86a1686ce4039985f508d9d498d6d 2023-03-22 15:55:41,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a6897edbdf5f4082bc08c40c9656705a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a6897edbdf5f4082bc08c40c9656705a 2023-03-22 15:55:41,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/831b9d914ea94e53ae7cad5158a98436 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/831b9d914ea94e53ae7cad5158a98436 2023-03-22 15:55:41,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a1606941f1e6490c808f683b5e5c13f3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family2/a1606941f1e6490c808f683b5e5c13f3 2023-03-22 15:55:41,182 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322501a3b8985bb454e96aae920c9280dc6 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230322501a3b8985bb454e96aae920c9280dc6 2023-03-22 15:55:41,183 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='ns:testMinorCompaction' column='family1' 2023-03-22 15:55:41,197 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:55:41,199 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39670, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:55:41,208 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:55:41,212 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34758, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:55:41,225 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230322/family1/4261fd739554422aa4dbe5c40a855def first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-22 15:55:41,226 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230322/family1/4261fd739554422aa4dbe5c40a855def}] 2023-03-22 15:55:41,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230322/family1/4261fd739554422aa4dbe5c40a855def for inclusion in 2c6ba5db98f0e0042380c0a57cebc9d0/family1 2023-03-22 15:55:41,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-22 15:55:41,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-22 15:55:41,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:55:41,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230322/family1/4261fd739554422aa4dbe5c40a855def to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__pre1di3jtvdgopju19jrhnfc6peq8u3sm8nn50n0dqig3m5ouvcpuaqgdvkjc78j/family1/4261fd739554422aa4dbe5c40a855def 2023-03-22 15:55:41,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__pre1di3jtvdgopju19jrhnfc6peq8u3sm8nn50n0dqig3m5ouvcpuaqgdvkjc78j/family1/4261fd739554422aa4dbe5c40a855def as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/524e6435405c40839673276ad83bcae9_SeqId_28_ 2023-03-22 15:55:41,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__pre1di3jtvdgopju19jrhnfc6peq8u3sm8nn50n0dqig3m5ouvcpuaqgdvkjc78j/family1/4261fd739554422aa4dbe5c40a855def into 2c6ba5db98f0e0042380c0a57cebc9d0/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/524e6435405c40839673276ad83bcae9_SeqId_28_ - updating store file list. 2023-03-22 15:55:41,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/524e6435405c40839673276ad83bcae9_SeqId_28_ into 2c6ba5db98f0e0042380c0a57cebc9d0/family1 2023-03-22 15:55:41,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__pre1di3jtvdgopju19jrhnfc6peq8u3sm8nn50n0dqig3m5ouvcpuaqgdvkjc78j/family1/4261fd739554422aa4dbe5c40a855def into 2c6ba5db98f0e0042380c0a57cebc9d0/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/family1/524e6435405c40839673276ad83bcae9_SeqId_28_) 2023-03-22 15:55:41,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__pre1di3jtvdgopju19jrhnfc6peq8u3sm8nn50n0dqig3m5ouvcpuaqgdvkjc78j/family1/4261fd739554422aa4dbe5c40a855def 2023-03-22 15:55:41,251 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='ns:testMinorCompaction' column='family1' 2023-03-22 15:55:41,252 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:41,253 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230322eb8f8b9ef71b423ea8271852764ddb96 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230322eb8f8b9ef71b423ea8271852764ddb96 2023-03-22 15:55:41,254 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-22 15:55:41,254 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230322' 2023-03-22 15:55:41,254 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230322 for table='ns:testMinorCompaction' and column='family1' 2023-03-22 15:55:41,574 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322e962dbb967a04f9b94bb31e630fe65ee to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230322e962dbb967a04f9b94bb31e630fe65ee 2023-03-22 15:55:41,600 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:55:41,600 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3eac4e78 to 127.0.0.1:58225 2023-03-22 15:55:41,600 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:41,601 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:55:41,601 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x121b7dad to 127.0.0.1:58225 2023-03-22 15:55:41,601 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:55:41,601 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='ns:testMinorCompaction' column='family1' 2023-03-22 15:55:41,613 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303222d6ca33c3a8a4ee0a46b69f27867f41a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303222d6ca33c3a8a4ee0a46b69f27867f41a 2023-03-22 15:55:41,614 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230322 table='ns:testMinorCompaction' column='family1' 2023-03-22 15:55:41,635 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230322/family1/9898560f9da14ba98f9544ce534e5d44 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-22 15:55:41,640 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230322/family1/9898560f9da14ba98f9544ce534e5d44}] 2023-03-22 15:55:41,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230322/family1/9898560f9da14ba98f9544ce534e5d44 for inclusion in 9ae79c6df98f35ced4ccb3fbc1d46164/family1 2023-03-22 15:55:41,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-22 15:55:41,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last=1 2023-03-22 15:55:41,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:55:41,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230322/family1/9898560f9da14ba98f9544ce534e5d44 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__hencmke66mgsmvokgm6bpsn6ufbmlk0293nsmokrk1mlhrknbit9hnnmmmfbeng5/family1/9898560f9da14ba98f9544ce534e5d44 2023-03-22 15:55:41,649 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230322/family1/b039a74f21314879bae7699818d2a051 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-22 15:55:41,650 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230322/family1/b039a74f21314879bae7699818d2a051}] 2023-03-22 15:55:41,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__hencmke66mgsmvokgm6bpsn6ufbmlk0293nsmokrk1mlhrknbit9hnnmmmfbeng5/family1/9898560f9da14ba98f9544ce534e5d44 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/3077616a8c5d461f892994f766ad7426_SeqId_28_ 2023-03-22 15:55:41,652 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230322/family1/b039a74f21314879bae7699818d2a051 for inclusion in 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1 2023-03-22 15:55:41,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__hencmke66mgsmvokgm6bpsn6ufbmlk0293nsmokrk1mlhrknbit9hnnmmmfbeng5/family1/9898560f9da14ba98f9544ce534e5d44 into 9ae79c6df98f35ced4ccb3fbc1d46164/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/3077616a8c5d461f892994f766ad7426_SeqId_28_ - updating store file list. 2023-03-22 15:55:41,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-22 15:55:41,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-22 15:55:41,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:55:41,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230322/family1/b039a74f21314879bae7699818d2a051 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__nd7t038sp26n39nitob21vb1pe0lkt5hpnoc6fskol8a4aumk58t2sktu2rmsifc/family1/b039a74f21314879bae7699818d2a051 2023-03-22 15:55:41,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/3077616a8c5d461f892994f766ad7426_SeqId_28_ into 9ae79c6df98f35ced4ccb3fbc1d46164/family1 2023-03-22 15:55:41,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__hencmke66mgsmvokgm6bpsn6ufbmlk0293nsmokrk1mlhrknbit9hnnmmmfbeng5/family1/9898560f9da14ba98f9544ce534e5d44 into 9ae79c6df98f35ced4ccb3fbc1d46164/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/family1/3077616a8c5d461f892994f766ad7426_SeqId_28_) 2023-03-22 15:55:41,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__hencmke66mgsmvokgm6bpsn6ufbmlk0293nsmokrk1mlhrknbit9hnnmmmfbeng5/family1/9898560f9da14ba98f9544ce534e5d44 2023-03-22 15:55:41,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__nd7t038sp26n39nitob21vb1pe0lkt5hpnoc6fskol8a4aumk58t2sktu2rmsifc/family1/b039a74f21314879bae7699818d2a051 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/4ee00e3276be48e99064999bdf819f44_SeqId_28_ 2023-03-22 15:55:41,658 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='ns:testMinorCompaction' column='family1' 2023-03-22 15:55:41,659 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__nd7t038sp26n39nitob21vb1pe0lkt5hpnoc6fskol8a4aumk58t2sktu2rmsifc/family1/b039a74f21314879bae7699818d2a051 into 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/4ee00e3276be48e99064999bdf819f44_SeqId_28_ - updating store file list. 2023-03-22 15:55:41,659 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:41,660 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230322825e7205812e4d02922e79029758e82d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230322825e7205812e4d02922e79029758e82d 2023-03-22 15:55:41,660 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-22 15:55:41,660 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230322' 2023-03-22 15:55:41,661 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230322 for table='ns:testMinorCompaction' and column='family1' 2023-03-22 15:55:41,662 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/4ee00e3276be48e99064999bdf819f44_SeqId_28_ into 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1 2023-03-22 15:55:41,662 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__nd7t038sp26n39nitob21vb1pe0lkt5hpnoc6fskol8a4aumk58t2sktu2rmsifc/family1/b039a74f21314879bae7699818d2a051 into 23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/family1/4ee00e3276be48e99064999bdf819f44_SeqId_28_) 2023-03-22 15:55:41,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__ns_testMinorCompaction__nd7t038sp26n39nitob21vb1pe0lkt5hpnoc6fskol8a4aumk58t2sktu2rmsifc/family1/b039a74f21314879bae7699818d2a051 2023-03-22 15:55:41,665 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230322 table='ns:testMinorCompaction' column='family1' 2023-03-22 15:55:41,665 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:41,666 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303222967290bb8564a7182e2128072da685e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303222967290bb8564a7182e2128072da685e 2023-03-22 15:55:41,666 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-22 15:55:41,666 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230322' 2023-03-22 15:55:41,667 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230322 for table='ns:testMinorCompaction' and column='family1' 2023-03-22 15:55:41,667 INFO [Listener at localhost/43289] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-22 15:55:41,669 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:41,671 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:41,675 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:41,675 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:41,699 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=256 (was 285), OpenFileDescriptor=639 (was 762), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=293 (was 364), ProcessCount=170 (was 170), AvailableMemoryMB=6291 (was 6297) 2023-03-22 15:55:41,716 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=256, OpenFileDescriptor=635, MaxFileDescriptor=60000, SystemLoadAverage=293, ProcessCount=170, AvailableMemoryMB=6290 2023-03-22 15:55:41,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-22 15:55:41,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:41,721 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:55:41,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-22 15:55:41,722 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:55:41,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-22 15:55:41,724 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:55:41,725 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc empty. 2023-03-22 15:55:41,726 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:55:41,726 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-22 15:55:41,733 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-22 15:55:41,734 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2873f0002a080f8edd78b8fa0916e3bc, NAME => 'testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:55:41,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-22 15:55:42,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-22 15:55:42,141 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:42,141 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 2873f0002a080f8edd78b8fa0916e3bc, disabling compactions & flushes 2023-03-22 15:55:42,141 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:42,141 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:42,141 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. after waiting 0 ms 2023-03-22 15:55:42,141 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:42,141 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:42,141 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:42,145 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:55:42,146 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679500542146"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500542146"}]},"ts":"1679500542146"} 2023-03-22 15:55:42,151 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-22 15:55:42,152 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:55:42,152 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500542152"}]},"ts":"1679500542152"} 2023-03-22 15:55:42,153 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-22 15:55:42,157 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2873f0002a080f8edd78b8fa0916e3bc, ASSIGN}] 2023-03-22 15:55:42,159 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2873f0002a080f8edd78b8fa0916e3bc, ASSIGN 2023-03-22 15:55:42,159 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2873f0002a080f8edd78b8fa0916e3bc, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:55:42,310 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2873f0002a080f8edd78b8fa0916e3bc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:42,311 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679500542310"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500542310"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500542310"}]},"ts":"1679500542310"} 2023-03-22 15:55:42,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 2873f0002a080f8edd78b8fa0916e3bc, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:42,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-22 15:55:42,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:42,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2873f0002a080f8edd78b8fa0916e3bc, NAME => 'testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc.', STARTKEY => '', ENDKEY => ''} 2023-03-22 15:55:42,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:55:42,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:42,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:55:42,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:55:42,474 INFO [StoreOpener-2873f0002a080f8edd78b8fa0916e3bc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:55:42,477 DEBUG [StoreOpener-2873f0002a080f8edd78b8fa0916e3bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1 2023-03-22 15:55:42,478 DEBUG [StoreOpener-2873f0002a080f8edd78b8fa0916e3bc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1 2023-03-22 15:55:42,478 INFO [StoreOpener-2873f0002a080f8edd78b8fa0916e3bc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2873f0002a080f8edd78b8fa0916e3bc columnFamilyName family1 2023-03-22 15:55:42,483 INFO [StoreOpener-2873f0002a080f8edd78b8fa0916e3bc-1] regionserver.HStore(310): Store=2873f0002a080f8edd78b8fa0916e3bc/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:42,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:55:42,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:55:42,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:55:42,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:55:42,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2873f0002a080f8edd78b8fa0916e3bc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67504892, jitterRate=0.0059012770652771}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-22 15:55:42,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:42,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc., pid=114, masterSystemTime=1679500542464 2023-03-22 15:55:42,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:42,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:42,491 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2873f0002a080f8edd78b8fa0916e3bc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:42,491 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679500542491"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500542491"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500542491"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500542491"}]},"ts":"1679500542491"} 2023-03-22 15:55:42,494 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-22 15:55:42,494 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 2873f0002a080f8edd78b8fa0916e3bc, server=jenkins-hbase4.apache.org,38637,1679500416185 in 181 msec 2023-03-22 15:55:42,495 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-22 15:55:42,495 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2873f0002a080f8edd78b8fa0916e3bc, ASSIGN in 337 msec 2023-03-22 15:55:42,496 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:55:42,496 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500542496"}]},"ts":"1679500542496"} 2023-03-22 15:55:42,497 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-22 15:55:42,499 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:55:42,500 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 780 msec 2023-03-22 15:55:42,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-22 15:55:42,826 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-22 15:55:42,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:42,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:42,854 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:42,854 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:42,854 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:42,854 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:42,854 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:42,855 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:42,856 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:42,856 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:42,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:42,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:42,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:42,857 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:42,857 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:42,857 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:42,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:42,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:42,857 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:42,858 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:42,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:42,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:42,858 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:42,859 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:42,859 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:42,859 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:42,859 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:42,860 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-22 15:55:43,300 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f5cdb6b2f6a14da2b98126539945c105 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130f5cdb6b2f6a14da2b98126539945c105 2023-03-22 15:55:43,305 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/6183353d56104ecc9abf6e63343324be 2023-03-22 15:55:43,309 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/6183353d56104ecc9abf6e63343324be as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/6183353d56104ecc9abf6e63343324be 2023-03-22 15:55:43,313 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/6183353d56104ecc9abf6e63343324be, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-22 15:55:43,314 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 453ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:43,314 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:43,314 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:43,314 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:43,314 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:43,314 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:43,314 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,315 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:43,315 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:43,316 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,316 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:43,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:43,317 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:43,317 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:43,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:43,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:43,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:43,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@342ffc57[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:43,318 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:43,318 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,321 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,322 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:43,322 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:43,322 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:43,322 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,322 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:43,323 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:43,323 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,323 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:43,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:43,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:43,323 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:43,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:43,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:43,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:43,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:43,326 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:43,326 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:43,326 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:43,326 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,326 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:43,328 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,328 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:43,328 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:43,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,328 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:43,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,333 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:43,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,335 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:43,335 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:43,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:43,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:43,335 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:43,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:55:43,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:43,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:43,335 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,336 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,336 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,336 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:43,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:43,336 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,336 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:43,336 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:43,336 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,336 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,436 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:43,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:43,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:43,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:43,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,452 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:43,452 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:43,452 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:43,452 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:43,452 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,452 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,455 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:43,455 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:43,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:43,455 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,455 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:43,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:43,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,456 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,460 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:55:43,460 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,460 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:43,460 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:43,460 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:43,460 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:43,460 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:43,460 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:43,460 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:43,460 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-22 15:55:43,482 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511286d430e827f8e4a28b5514242ee260baa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511286d430e827f8e4a28b5514242ee260baa 2023-03-22 15:55:43,893 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/43bc3d0a977243fba14228b32dd782aa 2023-03-22 15:55:43,897 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/43bc3d0a977243fba14228b32dd782aa as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/43bc3d0a977243fba14228b32dd782aa 2023-03-22 15:55:43,902 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/43bc3d0a977243fba14228b32dd782aa, entries=1, sequenceid=9, filesize=5.0 K 2023-03-22 15:55:43,904 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 444ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:43,904 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:43,904 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:43,904 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:43,904 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:43,904 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,905 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:43,905 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:43,906 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,906 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:43,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:43,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:43,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:43,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:43,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a3f765f[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:43,908 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:43,909 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,910 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,910 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:43,910 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:43,910 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:43,910 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,910 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:43,911 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,912 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:43,912 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:43,912 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:43,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:43,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:43,923 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:43,923 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:43,923 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:43,923 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,923 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:43,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:43,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:43,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,926 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:43,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:43,927 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:43,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,928 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:43,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,930 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:43,930 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:43,930 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,930 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,931 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,931 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:43,931 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,931 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,931 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:43,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:43,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:43,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:43,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:43,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:55:43,931 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:43,931 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:43,931 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:43,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:43,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:44,031 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:44,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:44,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:44,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:44,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,038 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:44,038 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:44,038 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:44,038 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:44,039 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,039 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,043 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,043 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:44,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:44,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:44,043 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,043 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:44,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:44,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,044 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:55:44,044 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,044 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:44,044 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:44,044 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:44,045 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:44,045 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:44,045 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:44,045 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:44,045 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:55:44,070 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201cdfc5c91c0534b9493ab75712d622cd3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201cdfc5c91c0534b9493ab75712d622cd3 2023-03-22 15:55:44,085 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/b6f3d3616e574ef885e586c2b666134f 2023-03-22 15:55:44,093 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/b6f3d3616e574ef885e586c2b666134f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/b6f3d3616e574ef885e586c2b666134f 2023-03-22 15:55:44,099 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/b6f3d3616e574ef885e586c2b666134f, entries=1, sequenceid=13, filesize=4.9 K 2023-03-22 15:55:44,100 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 55ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:44,100 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:44,100 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:44,100 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:44,100 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:44,100 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,100 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:44,100 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:44,104 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:44,105 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,105 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,105 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:44,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:44,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d7c4740[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:44,106 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:44,106 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,107 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,108 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:44,108 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:44,108 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:44,108 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,108 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:44,110 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:44,110 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:44,110 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,110 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:44,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:44,113 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:44,113 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:44,113 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:44,113 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,113 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:44,118 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:44,120 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,120 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:44,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,120 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:44,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,122 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:44,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,126 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:44,126 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:44,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:44,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:44,126 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,126 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,126 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:44,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:44,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:44,126 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,126 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,126 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,126 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,127 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,127 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:44,127 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:44,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:55:44,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:44,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:44,227 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:44,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:44,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:44,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:44,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,236 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:44,236 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:44,236 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:44,236 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:44,237 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,237 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,238 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,238 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:44,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:44,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:44,238 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,238 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:44,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,239 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:44,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:55:44,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,239 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:44,239 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:44,239 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:44,244 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:44,244 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:44,244 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:44,244 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:44,244 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:55:44,262 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e04120de2a7e4e68b4b750b3ace0ebfd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205e04120de2a7e4e68b4b750b3ace0ebfd 2023-03-22 15:55:44,277 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/1d782c2a854446a6ac0b25334d54c628 2023-03-22 15:55:44,281 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/1d782c2a854446a6ac0b25334d54c628 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d782c2a854446a6ac0b25334d54c628 2023-03-22 15:55:44,284 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d782c2a854446a6ac0b25334d54c628, entries=1, sequenceid=17, filesize=4.9 K 2023-03-22 15:55:44,285 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 41ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:44,285 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:44,285 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:44,285 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:44,285 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:44,285 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,285 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:44,285 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:44,287 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,287 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:44,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:44,287 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,287 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:44,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:44,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:44,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:44,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:44,291 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46f644c9[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:44,291 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:44,291 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,292 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:44,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:44,292 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:44,293 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,293 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:44,298 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,298 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:44,299 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:44,299 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:44,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:44,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:44,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:44,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:44,303 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:44,303 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:44,303 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:44,303 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,303 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:44,305 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,305 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:44,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,305 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:44,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:44,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,306 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:44,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,307 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:44,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:44,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,311 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:44,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:44,311 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:44,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:55:44,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:44,411 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:44,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:44,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:44,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:44,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,418 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:44,418 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:44,418 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:44,418 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:44,419 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,419 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,420 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:44,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:44,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:44,420 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,421 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,421 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:44,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,421 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,421 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:55:44,421 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,422 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:44,422 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:44,422 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:44,423 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:44,423 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:44,423 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:44,423 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:44,423 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:55:44,461 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512284f82ea5e4f8744f8b89eb725a887da0c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512284f82ea5e4f8744f8b89eb725a887da0c 2023-03-22 15:55:44,874 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/ac818950cb414db3be611c7d541d546d 2023-03-22 15:55:44,882 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/ac818950cb414db3be611c7d541d546d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ac818950cb414db3be611c7d541d546d 2023-03-22 15:55:44,887 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ac818950cb414db3be611c7d541d546d, entries=1, sequenceid=21, filesize=4.9 K 2023-03-22 15:55:44,887 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 464ms, sequenceid=21, compaction requested=true 2023-03-22 15:55:44,888 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:44,888 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:44,888 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:44,888 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:44,888 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,888 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:44,888 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:44,890 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:44,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:44,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:44,893 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:44,893 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@df0d504[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:44,893 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:44,893 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,900 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,901 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,901 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:44,901 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:44,901 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:44,901 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:44,902 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,902 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:44,902 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:44,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:44,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:44,903 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:44,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:44,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:44,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:44,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:44,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:44,906 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:44,906 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:44,906 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:44,906 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,906 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:44,908 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:44,909 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,909 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,909 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:44,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,911 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:44,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:44,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:44,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:44,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:44,921 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:44,921 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:44,922 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:44,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:55:44,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:45,022 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:45,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:45,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:45,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:45,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,029 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:45,029 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:45,029 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:45,029 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:45,030 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,030 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,031 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:45,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:45,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,032 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,032 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,032 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:45,036 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:45,036 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:45,036 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:45,036 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:55:45,455 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231084d7e314cd14bd29611525c1545487f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231084d7e314cd14bd29611525c1545487f 2023-03-22 15:55:45,463 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/ab9515e87dde44d2acac0c963b3f3f58 2023-03-22 15:55:45,469 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/ab9515e87dde44d2acac0c963b3f3f58 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ab9515e87dde44d2acac0c963b3f3f58 2023-03-22 15:55:45,473 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ab9515e87dde44d2acac0c963b3f3f58, entries=1, sequenceid=25, filesize=4.9 K 2023-03-22 15:55:45,474 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 438ms, sequenceid=25, compaction requested=true 2023-03-22 15:55:45,474 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:45,474 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:45,474 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:45,475 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:45,475 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,475 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:45,475 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:45,476 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:45,476 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:45,477 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,477 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:45,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:45,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:45,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:45,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:45,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4cbcd76b[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:45,479 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:45,479 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,481 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,481 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:45,481 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:45,481 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:45,481 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,481 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:45,483 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,483 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:45,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:45,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:45,483 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:45,484 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:45,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:45,486 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:45,486 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:45,486 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:45,486 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,486 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:45,488 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,488 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,488 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:45,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:45,489 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:45,490 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,494 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,494 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,494 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:45,494 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:45,494 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,494 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,494 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:45,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:45,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:45,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:55:45,494 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:45,494 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:45,495 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:45,494 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:45,495 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:45,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:45,595 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:45,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:45,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:45,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:45,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,601 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:45,601 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:45,601 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:45,601 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:45,602 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,602 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,603 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:45,603 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:45,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:45,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,604 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,604 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:45,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:45,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,607 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:55:45,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,607 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:45,607 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:45,607 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:45,607 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:45,607 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:45,607 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:45,607 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:45,609 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:55:45,625 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601012c4824f1dd994805a0aea1bac92fdfc5 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601012c4824f1dd994805a0aea1bac92fdfc5 2023-03-22 15:55:45,631 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/ec625995e2c04695960c6e526b738963 2023-03-22 15:55:45,636 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/ec625995e2c04695960c6e526b738963 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ec625995e2c04695960c6e526b738963 2023-03-22 15:55:45,641 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ec625995e2c04695960c6e526b738963, entries=1, sequenceid=29, filesize=4.9 K 2023-03-22 15:55:45,642 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 33ms, sequenceid=29, compaction requested=true 2023-03-22 15:55:45,642 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:45,642 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:45,645 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:45,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:45,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:45,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:45,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:45,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:45,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6355cffb[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:45,647 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:45,647 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,647 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,647 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,648 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:45,649 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,649 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:45,649 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:45,649 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:45,649 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,649 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:45,650 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,650 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:45,650 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:45,650 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:45,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:45,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:45,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:45,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:45,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:45,653 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:45,653 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:45,653 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:45,653 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:45,653 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,653 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:45,656 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,656 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:45,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:45,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,656 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:45,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:45,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:45,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:45,656 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,658 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:45,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,663 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:45,664 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:45,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:45,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:45,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:45,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:45,664 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,664 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,664 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,664 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,664 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:45,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:45,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:45,664 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,664 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:45,665 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:45,665 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:45,665 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,666 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:55:45,765 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:45,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:45,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:45,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:45,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,772 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:45,772 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:45,772 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:45,772 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:45,772 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,773 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,774 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:45,774 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:45,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:45,775 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,775 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:45,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:45,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:55:45,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,776 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:45,776 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:45,776 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:45,776 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:45,776 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:45,776 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:45,776 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:45,777 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:55:45,790 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037628b0f78eff4159bcd5608e91fe701d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601037628b0f78eff4159bcd5608e91fe701d 2023-03-22 15:55:45,797 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/19540153b3b24cacaf8b7e0e6260ab34 2023-03-22 15:55:45,801 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/19540153b3b24cacaf8b7e0e6260ab34 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/19540153b3b24cacaf8b7e0e6260ab34 2023-03-22 15:55:45,805 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/19540153b3b24cacaf8b7e0e6260ab34, entries=1, sequenceid=33, filesize=4.9 K 2023-03-22 15:55:45,807 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 30ms, sequenceid=33, compaction requested=true 2023-03-22 15:55:45,807 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:45,807 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:45,807 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:45,807 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:45,807 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,807 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:45,807 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:45,809 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,809 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:45,810 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,810 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:45,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:45,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17f11f54[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:45,811 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:45,811 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,813 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,813 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:45,813 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:45,813 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:45,813 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,813 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:45,816 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,816 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:45,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:45,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:45,816 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:45,816 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:45,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:45,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:45,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:45,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:45,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:45,818 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:45,818 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:45,818 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:45,818 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,818 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:45,820 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:45,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:45,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:45,821 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,821 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:45,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,821 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:45,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:45,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:45,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:45,822 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,827 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:45,827 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:45,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:45,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:45,827 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:45,827 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:45,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:45,827 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,828 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,828 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,828 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,828 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,828 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:45,828 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:45,828 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:45,928 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:45,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:45,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:45,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:45,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,934 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:45,934 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:45,934 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:45,934 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:45,935 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,935 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,937 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:45,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:45,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:45,937 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:45,937 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:45,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:45,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:55:45,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:45,938 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:45,938 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:45,938 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:45,938 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:45,938 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:45,938 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:45,938 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:45,938 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:55:45,954 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907259bd6d5312b4f1c8dcc5c559f174723 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907259bd6d5312b4f1c8dcc5c559f174723 2023-03-22 15:55:46,362 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/d1368fde61c643ef8c337d4541565a9e 2023-03-22 15:55:46,366 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/d1368fde61c643ef8c337d4541565a9e as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/d1368fde61c643ef8c337d4541565a9e 2023-03-22 15:55:46,370 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/d1368fde61c643ef8c337d4541565a9e, entries=1, sequenceid=37, filesize=4.9 K 2023-03-22 15:55:46,371 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 433ms, sequenceid=37, compaction requested=true 2023-03-22 15:55:46,371 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:46,371 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:46,371 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:46,371 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:46,371 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,372 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:46,372 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:46,373 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,373 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:46,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:46,374 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,374 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:46,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:46,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:46,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:46,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:46,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c22b05b[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:46,375 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:46,375 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,376 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,377 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:46,377 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:46,377 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:46,377 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,377 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:46,378 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,378 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:46,378 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:46,378 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:46,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:46,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:46,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:46,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:46,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:46,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:46,382 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:46,382 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:46,382 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:46,382 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,382 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:46,386 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,387 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,387 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:46,387 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:46,389 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:46,391 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:46,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:46,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:46,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:46,391 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:46,392 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:46,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:46,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:46,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-22 15:55:46,492 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:46,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:46,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:46,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:46,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,498 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:46,498 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:46,498 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-22 15:55:46,498 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-22 15:55:46,499 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,499 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,500 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,500 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,500 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:46,500 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:46,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:46,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:46,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:46,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,502 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,502 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-22 15:55:46,502 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,502 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-22 15:55:46,502 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:46,502 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-22 15:55:46,503 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-22 15:55:46,503 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:46,503 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:46,503 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. started... 2023-03-22 15:55:46,503 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2873f0002a080f8edd78b8fa0916e3bc 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-22 15:55:46,520 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511202e1c0d37af254f8aa6e349c332ed751b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511202e1c0d37af254f8aa6e349c332ed751b 2023-03-22 15:55:46,527 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/c19365e505de454ebbae73033dad6a0b 2023-03-22 15:55:46,531 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/c19365e505de454ebbae73033dad6a0b as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/c19365e505de454ebbae73033dad6a0b 2023-03-22 15:55:46,537 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/c19365e505de454ebbae73033dad6a0b, entries=1, sequenceid=41, filesize=4.9 K 2023-03-22 15:55:46,538 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2873f0002a080f8edd78b8fa0916e3bc in 35ms, sequenceid=41, compaction requested=true 2023-03-22 15:55:46,538 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:46,538 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:46,538 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-22 15:55:46,538 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-22 15:55:46,538 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,538 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-22 15:55:46,538 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-22 15:55:46,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,540 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:46,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:46,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,540 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:46,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:46,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:46,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:46,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-22 15:55:46,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@360f2b3b[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:46,543 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-22 15:55:46,543 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,546 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,546 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,546 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:46,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-22 15:55:46,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-22 15:55:46,546 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:46,547 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-22 15:55:46,547 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,548 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:46,548 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-22 15:55:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:46,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:46,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:46,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-22 15:55:46,550 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-22 15:55:46,550 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:46,550 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:46,550 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,550 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:46,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:46,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:46,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,552 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:46,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:46,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,553 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:46,554 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:46,558 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-22 15:55:46,558 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:46,558 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:46,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:46,558 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:46,559 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:46,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:46,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:46,659 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:46,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:46,661 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-22 15:55:46,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:55:46,663 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:55:46,664 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501146664 2023-03-22 15:55:46,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:55:46,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-22 15:55:46,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-22 15:55:46,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:55:46,666 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x539ba0d1 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:55:46,675 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4302b5fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:55:46,675 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:55:47,109 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034f55a4f5ee044448928fa9cd159123e5 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601034f55a4f5ee044448928fa9cd159123e5 2023-03-22 15:55:47,110 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:55:47,128 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:55:47,130 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43636, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:55:47,137 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:55:47,138 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52528, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:55:47,152 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2edaa6b67e6244dbb4ee622386d47dfa first=Optional[r5] last=Optional[r6] 2023-03-22 15:55:47,153 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2edaa6b67e6244dbb4ee622386d47dfa}] 2023-03-22 15:55:47,158 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2edaa6b67e6244dbb4ee622386d47dfa for inclusion in 2873f0002a080f8edd78b8fa0916e3bc/family1 2023-03-22 15:55:47,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-22 15:55:47,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:55:47,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:47,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2edaa6b67e6244dbb4ee622386d47dfa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lf9a8d1596ncv8gvbrkpod3mgdoru6ejj23ctm66in09f2d81dh4j05uc7srisrc/family1/2edaa6b67e6244dbb4ee622386d47dfa 2023-03-22 15:55:47,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lf9a8d1596ncv8gvbrkpod3mgdoru6ejj23ctm66in09f2d81dh4j05uc7srisrc/family1/2edaa6b67e6244dbb4ee622386d47dfa as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d1209576f8341d882aa7795b290a280_SeqId_44_ 2023-03-22 15:55:47,170 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lf9a8d1596ncv8gvbrkpod3mgdoru6ejj23ctm66in09f2d81dh4j05uc7srisrc/family1/2edaa6b67e6244dbb4ee622386d47dfa into 2873f0002a080f8edd78b8fa0916e3bc/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d1209576f8341d882aa7795b290a280_SeqId_44_ - updating store file list. 2023-03-22 15:55:47,175 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d1209576f8341d882aa7795b290a280_SeqId_44_ into 2873f0002a080f8edd78b8fa0916e3bc/family1 2023-03-22 15:55:47,175 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lf9a8d1596ncv8gvbrkpod3mgdoru6ejj23ctm66in09f2d81dh4j05uc7srisrc/family1/2edaa6b67e6244dbb4ee622386d47dfa into 2873f0002a080f8edd78b8fa0916e3bc/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d1209576f8341d882aa7795b290a280_SeqId_44_) 2023-03-22 15:55:47,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lf9a8d1596ncv8gvbrkpod3mgdoru6ejj23ctm66in09f2d81dh4j05uc7srisrc/family1/2edaa6b67e6244dbb4ee622386d47dfa 2023-03-22 15:55:47,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-22 15:55:47,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-22 15:55:47,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(1912): 2873f0002a080f8edd78b8fa0916e3bc/family1 is initiating minor compaction 2023-03-22 15:55:47,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4bc144a7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:55:47,179 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2873f0002a080f8edd78b8fa0916e3bc/family1 in testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:55:47,179 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/43bc3d0a977243fba14228b32dd782aa, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/b6f3d3616e574ef885e586c2b666134f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d782c2a854446a6ac0b25334d54c628, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ac818950cb414db3be611c7d541d546d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ab9515e87dde44d2acac0c963b3f3f58, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ec625995e2c04695960c6e526b738963, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/19540153b3b24cacaf8b7e0e6260ab34, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/d1368fde61c643ef8c337d4541565a9e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/c19365e505de454ebbae73033dad6a0b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d1209576f8341d882aa7795b290a280_SeqId_44_] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp, totalSize=49.7 K 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 43bc3d0a977243fba14228b32dd782aa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting b6f3d3616e574ef885e586c2b666134f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 1d782c2a854446a6ac0b25334d54c628, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting ac818950cb414db3be611c7d541d546d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting ab9515e87dde44d2acac0c963b3f3f58, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting ec625995e2c04695960c6e526b738963, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 19540153b3b24cacaf8b7e0e6260ab34, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting d1368fde61c643ef8c337d4541565a9e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting c19365e505de454ebbae73033dad6a0b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-22 15:55:47,179 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] compactions.Compactor(207): Compacting 1d1209576f8341d882aa7795b290a280_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-22 15:55:47,181 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:55:47,182 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:47,185 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601012c4824f1dd994805a0aea1bac92fdfc5 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601012c4824f1dd994805a0aea1bac92fdfc5 2023-03-22 15:55:47,187 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601037628b0f78eff4159bcd5608e91fe701d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601037628b0f78eff4159bcd5608e91fe701d 2023-03-22 15:55:47,187 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-22 15:55:47,187 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-22 15:55:47,188 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:55:47,188 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:55:47,228 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2873f0002a080f8edd78b8fa0916e3bc#family1#compaction#166 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-03-22 15:55:47,649 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316655d27f213d4910a59d42539c0c4ac0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316655d27f213d4910a59d42539c0c4ac0 2023-03-22 15:55:47,650 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:55:47,673 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bb4c6dbd82204850bf6f042b103631bd first=Optional[r1] last=Optional[r4] 2023-03-22 15:55:47,673 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bb4c6dbd82204850bf6f042b103631bd}] 2023-03-22 15:55:47,675 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bb4c6dbd82204850bf6f042b103631bd for inclusion in 2873f0002a080f8edd78b8fa0916e3bc/family1 2023-03-22 15:55:47,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-22 15:55:47,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:55:47,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:47,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bb4c6dbd82204850bf6f042b103631bd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__vrlcj361f00cf4ev17vnvgbp4j5hndvraur28avk3dikuuee2horo0v8u6g68nrp/family1/bb4c6dbd82204850bf6f042b103631bd 2023-03-22 15:55:47,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__vrlcj361f00cf4ev17vnvgbp4j5hndvraur28avk3dikuuee2horo0v8u6g68nrp/family1/bb4c6dbd82204850bf6f042b103631bd as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/f2dc8a622dd54c6c8cb74c71bc38e569_SeqId_46_ 2023-03-22 15:55:47,680 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__vrlcj361f00cf4ev17vnvgbp4j5hndvraur28avk3dikuuee2horo0v8u6g68nrp/family1/bb4c6dbd82204850bf6f042b103631bd into 2873f0002a080f8edd78b8fa0916e3bc/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/f2dc8a622dd54c6c8cb74c71bc38e569_SeqId_46_ - updating store file list. 2023-03-22 15:55:47,687 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/f2dc8a622dd54c6c8cb74c71bc38e569_SeqId_46_ into 2873f0002a080f8edd78b8fa0916e3bc/family1 2023-03-22 15:55:47,687 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__vrlcj361f00cf4ev17vnvgbp4j5hndvraur28avk3dikuuee2horo0v8u6g68nrp/family1/bb4c6dbd82204850bf6f042b103631bd into 2873f0002a080f8edd78b8fa0916e3bc/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/f2dc8a622dd54c6c8cb74c71bc38e569_SeqId_46_) 2023-03-22 15:55:47,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__vrlcj361f00cf4ev17vnvgbp4j5hndvraur28avk3dikuuee2horo0v8u6g68nrp/family1/bb4c6dbd82204850bf6f042b103631bd 2023-03-22 15:55:47,690 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:55:47,690 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:47,692 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201cdfc5c91c0534b9493ab75712d622cd3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201cdfc5c91c0534b9493ab75712d622cd3 2023-03-22 15:55:47,693 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205e04120de2a7e4e68b4b750b3ace0ebfd to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205e04120de2a7e4e68b4b750b3ace0ebfd 2023-03-22 15:55:47,694 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512284f82ea5e4f8744f8b89eb725a887da0c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512284f82ea5e4f8744f8b89eb725a887da0c 2023-03-22 15:55:47,696 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231084d7e314cd14bd29611525c1545487f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231084d7e314cd14bd29611525c1545487f 2023-03-22 15:55:47,696 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-22 15:55:47,696 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-22 15:55:47,697 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:55:47,697 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:55:47,752 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511301a029e44b52a4d41a89a46cc8db0d698 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511301a029e44b52a4d41a89a46cc8db0d698 2023-03-22 15:55:47,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:55:47,784 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/03e9cabddc8f412a993fb5d447aba9b6 first=Optional[r00] last=Optional[r8] 2023-03-22 15:55:47,786 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/03e9cabddc8f412a993fb5d447aba9b6}] 2023-03-22 15:55:47,788 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/03e9cabddc8f412a993fb5d447aba9b6 for inclusion in 2873f0002a080f8edd78b8fa0916e3bc/family1 2023-03-22 15:55:47,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-22 15:55:47,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last= 2023-03-22 15:55:47,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:47,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/03e9cabddc8f412a993fb5d447aba9b6 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n8jacp4q5pbsnjlpsrsg2km19rgnbgehhd1td96skuvq5477nmnbkqderi20ude9/family1/03e9cabddc8f412a993fb5d447aba9b6 2023-03-22 15:55:47,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n8jacp4q5pbsnjlpsrsg2km19rgnbgehhd1td96skuvq5477nmnbkqderi20ude9/family1/03e9cabddc8f412a993fb5d447aba9b6 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/4c6a464210a14da8b57fff46bd4dda2f_SeqId_48_ 2023-03-22 15:55:47,796 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n8jacp4q5pbsnjlpsrsg2km19rgnbgehhd1td96skuvq5477nmnbkqderi20ude9/family1/03e9cabddc8f412a993fb5d447aba9b6 into 2873f0002a080f8edd78b8fa0916e3bc/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/4c6a464210a14da8b57fff46bd4dda2f_SeqId_48_ - updating store file list. 2023-03-22 15:55:47,799 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/4c6a464210a14da8b57fff46bd4dda2f_SeqId_48_ into 2873f0002a080f8edd78b8fa0916e3bc/family1 2023-03-22 15:55:47,800 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n8jacp4q5pbsnjlpsrsg2km19rgnbgehhd1td96skuvq5477nmnbkqderi20ude9/family1/03e9cabddc8f412a993fb5d447aba9b6 into 2873f0002a080f8edd78b8fa0916e3bc/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/4c6a464210a14da8b57fff46bd4dda2f_SeqId_48_) 2023-03-22 15:55:47,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMinorCompactionWithMonthlyPolicy__n8jacp4q5pbsnjlpsrsg2km19rgnbgehhd1td96skuvq5477nmnbkqderi20ude9/family1/03e9cabddc8f412a993fb5d447aba9b6 2023-03-22 15:55:47,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] compactions.SortedCompactionPolicy(75): Selecting compaction from 13 store files, 10 compacting, 2 eligible, 16 blocking 2023-03-22 15:55:47,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-22 15:55:47,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-22 15:55:47,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.CompactSplit(415): Not compacting testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. because compaction request was cancelled 2023-03-22 15:55:47,803 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:55:47,803 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:47,805 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511202e1c0d37af254f8aa6e349c332ed751b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511202e1c0d37af254f8aa6e349c332ed751b 2023-03-22 15:55:47,806 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511286d430e827f8e4a28b5514242ee260baa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511286d430e827f8e4a28b5514242ee260baa 2023-03-22 15:55:47,808 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130f5cdb6b2f6a14da2b98126539945c105 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130f5cdb6b2f6a14da2b98126539945c105 2023-03-22 15:55:47,808 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-22 15:55:47,808 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-22 15:55:47,808 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:55:47,809 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:55:47,809 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-22 15:55:47,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-22 15:55:47,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:55:47,811 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-22 15:55:47,813 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.1490 sec 2023-03-22 15:55:48,015 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:55:48,044 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/.tmp/family1/0352671bed554acdac5a3cede3b3a2c7 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/0352671bed554acdac5a3cede3b3a2c7 2023-03-22 15:55:48,051 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2873f0002a080f8edd78b8fa0916e3bc/family1 of 2873f0002a080f8edd78b8fa0916e3bc into 0352671bed554acdac5a3cede3b3a2c7(size=6.6 K), total size for store is 344.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:48,051 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:55:48,051 INFO [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1679500547178; duration=0sec 2023-03-22 15:55:48,051 DEBUG [RS:0;jenkins-hbase4:38637-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-22 15:55:48,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/43bc3d0a977243fba14228b32dd782aa, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/b6f3d3616e574ef885e586c2b666134f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d782c2a854446a6ac0b25334d54c628, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ac818950cb414db3be611c7d541d546d, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ab9515e87dde44d2acac0c963b3f3f58, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ec625995e2c04695960c6e526b738963, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/19540153b3b24cacaf8b7e0e6260ab34, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/d1368fde61c643ef8c337d4541565a9e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/c19365e505de454ebbae73033dad6a0b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d1209576f8341d882aa7795b290a280_SeqId_44_] to archive 2023-03-22 15:55:48,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:48,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/43bc3d0a977243fba14228b32dd782aa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/43bc3d0a977243fba14228b32dd782aa 2023-03-22 15:55:48,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/b6f3d3616e574ef885e586c2b666134f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/b6f3d3616e574ef885e586c2b666134f 2023-03-22 15:55:48,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d782c2a854446a6ac0b25334d54c628 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d782c2a854446a6ac0b25334d54c628 2023-03-22 15:55:48,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ac818950cb414db3be611c7d541d546d to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ac818950cb414db3be611c7d541d546d 2023-03-22 15:55:48,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ab9515e87dde44d2acac0c963b3f3f58 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ab9515e87dde44d2acac0c963b3f3f58 2023-03-22 15:55:48,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ec625995e2c04695960c6e526b738963 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/ec625995e2c04695960c6e526b738963 2023-03-22 15:55:48,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/19540153b3b24cacaf8b7e0e6260ab34 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/19540153b3b24cacaf8b7e0e6260ab34 2023-03-22 15:55:48,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/d1368fde61c643ef8c337d4541565a9e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/d1368fde61c643ef8c337d4541565a9e 2023-03-22 15:55:48,135 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/c19365e505de454ebbae73033dad6a0b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/c19365e505de454ebbae73033dad6a0b 2023-03-22 15:55:48,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d1209576f8341d882aa7795b290a280_SeqId_44_ to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/family1/1d1209576f8341d882aa7795b290a280_SeqId_44_ 2023-03-22 15:55:48,139 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=266 (was 256) - Thread LEAK? -, OpenFileDescriptor=670 (was 635) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=318 (was 293) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6279 (was 6290) 2023-03-22 15:55:48,154 INFO [Listener at localhost/43289] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=266, OpenFileDescriptor=670, MaxFileDescriptor=60000, SystemLoadAverage=318, ProcessCount=170, AvailableMemoryMB=6279 2023-03-22 15:55:48,176 DEBUG [Listener at localhost/43289] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-22 15:55:48,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xA7P\x93V-'\x02\x15\xD9\xC7\xE0\xB9\xCB\x1C^\x0D"\x10\x81I\x1B\xBBw\xBD\x00{\x1BR\x8F\xFBf\xDE\xD7\xC7*\x10\xF7\x14U\x1F@"\x8F;\xA7\xBA\xF2^\x94\xCC\xA9\xDD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-22 15:55:48,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-22 15:55:48,255 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-22 15:55:48,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-22 15:55:48,256 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-22 15:55:48,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-22 15:55:48,264 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:48,264 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:48,272 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa empty. 2023-03-22 15:55:48,272 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:48,272 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f empty. 2023-03-22 15:55:48,273 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa empty. 2023-03-22 15:55:48,273 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:48,273 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:48,273 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:48,273 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-22 15:55:48,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-22 15:55:48,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-22 15:55:48,694 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-22 15:55:48,695 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => eb17b8df8962f746a288d51c9615b0aa, NAME => 'testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xA7P\x93V-'\x02\x15\xD9\xC7\xE0\xB9\xCB\x1C^\x0D"\x10\x81I\x1B\xBBw\xBD\x00{\x1BR\x8F\xFBf\xDE\xD7\xC7*\x10\xF7\x14U\x1F@"\x8F;\xA7\xBA\xF2^\x94\xCC\xA9\xDD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:55:48,696 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 50974b1aa79ca5843b01217d351ab6aa, NAME => 'testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xA7P\x93V-'\x02\x15\xD9\xC7\xE0\xB9\xCB\x1C^\x0D"\x10\x81I\x1B\xBBw\xBD\x00{\x1BR\x8F\xFBf\xDE\xD7\xC7*\x10\xF7\x14U\x1F@"\x8F;\xA7\xBA\xF2^\x94\xCC\xA9\xDD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:55:48,696 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 9e2e5b3b5959941e4d2218f3d3ff804f, NAME => 'testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xA7P\x93V-'\x02\x15\xD9\xC7\xE0\xB9\xCB\x1C^\x0D"\x10\x81I\x1B\xBBw\xBD\x00{\x1BR\x8F\xFBf\xDE\xD7\xC7*\x10\xF7\x14U\x1F@"\x8F;\xA7\xBA\xF2^\x94\xCC\xA9\xDD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/.tmp 2023-03-22 15:55:48,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-22 15:55:49,113 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:49,114 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 50974b1aa79ca5843b01217d351ab6aa, disabling compactions & flushes 2023-03-22 15:55:49,114 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:49,114 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:49,114 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. after waiting 0 ms 2023-03-22 15:55:49,114 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:49,114 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:49,114 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:55:49,115 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:49,115 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing eb17b8df8962f746a288d51c9615b0aa, disabling compactions & flushes 2023-03-22 15:55:49,115 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:49,115 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:49,115 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. after waiting 0 ms 2023-03-22 15:55:49,115 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:49,115 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:49,115 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:55:49,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:49,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 9e2e5b3b5959941e4d2218f3d3ff804f, disabling compactions & flushes 2023-03-22 15:55:49,118 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:49,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:49,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. after waiting 0 ms 2023-03-22 15:55:49,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:49,118 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:49,118 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:55:49,120 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-22 15:55:49,121 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500549121"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500549121"}]},"ts":"1679500549121"} 2023-03-22 15:55:49,121 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500549121"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500549121"}]},"ts":"1679500549121"} 2023-03-22 15:55:49,121 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679500549121"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679500549121"}]},"ts":"1679500549121"} 2023-03-22 15:55:49,123 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-22 15:55:49,123 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-22 15:55:49,124 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500549123"}]},"ts":"1679500549123"} 2023-03-22 15:55:49,124 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-22 15:55:49,128 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=50974b1aa79ca5843b01217d351ab6aa, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=9e2e5b3b5959941e4d2218f3d3ff804f, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=eb17b8df8962f746a288d51c9615b0aa, ASSIGN}] 2023-03-22 15:55:49,129 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=9e2e5b3b5959941e4d2218f3d3ff804f, ASSIGN 2023-03-22 15:55:49,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=50974b1aa79ca5843b01217d351ab6aa, ASSIGN 2023-03-22 15:55:49,130 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=eb17b8df8962f746a288d51c9615b0aa, ASSIGN 2023-03-22 15:55:49,130 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=9e2e5b3b5959941e4d2218f3d3ff804f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:55:49,130 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=50974b1aa79ca5843b01217d351ab6aa, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:55:49,130 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=eb17b8df8962f746a288d51c9615b0aa, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38637,1679500416185; forceNewPlan=false, retain=false 2023-03-22 15:55:49,282 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=50974b1aa79ca5843b01217d351ab6aa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:49,282 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=9e2e5b3b5959941e4d2218f3d3ff804f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:49,282 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500549282"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500549282"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500549282"}]},"ts":"1679500549282"} 2023-03-22 15:55:49,282 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=eb17b8df8962f746a288d51c9615b0aa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:49,282 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679500549282"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500549282"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500549282"}]},"ts":"1679500549282"} 2023-03-22 15:55:49,283 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500549282"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679500549282"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679500549282"}]},"ts":"1679500549282"} 2023-03-22 15:55:49,284 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure 50974b1aa79ca5843b01217d351ab6aa, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:49,285 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure 9e2e5b3b5959941e4d2218f3d3ff804f, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:49,285 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure eb17b8df8962f746a288d51c9615b0aa, server=jenkins-hbase4.apache.org,38637,1679500416185}] 2023-03-22 15:55:49,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-22 15:55:49,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:49,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9e2e5b3b5959941e4d2218f3d3ff804f, NAME => 'testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f.', STARTKEY => '1', ENDKEY => '2'} 2023-03-22 15:55:49,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:49,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:49,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:49,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:49,462 INFO [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:49,463 DEBUG [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1 2023-03-22 15:55:49,463 DEBUG [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1 2023-03-22 15:55:49,464 INFO [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9e2e5b3b5959941e4d2218f3d3ff804f columnFamilyName family1 2023-03-22 15:55:49,465 INFO [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] regionserver.HStore(310): Store=9e2e5b3b5959941e4d2218f3d3ff804f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:49,465 INFO [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:49,466 DEBUG [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2 2023-03-22 15:55:49,467 DEBUG [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2 2023-03-22 15:55:49,467 INFO [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9e2e5b3b5959941e4d2218f3d3ff804f columnFamilyName family2 2023-03-22 15:55:49,468 INFO [StoreOpener-9e2e5b3b5959941e4d2218f3d3ff804f-1] regionserver.HStore(310): Store=9e2e5b3b5959941e4d2218f3d3ff804f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:49,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:49,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:49,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:55:49,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:55:49,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:55:49,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9e2e5b3b5959941e4d2218f3d3ff804f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61364881, jitterRate=-0.08559201657772064}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:55:49,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:55:49,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f., pid=121, masterSystemTime=1679500549436 2023-03-22 15:55:49,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:49,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:49,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:49,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eb17b8df8962f746a288d51c9615b0aa, NAME => 'testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa.', STARTKEY => '2', ENDKEY => ''} 2023-03-22 15:55:49,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:49,477 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=9e2e5b3b5959941e4d2218f3d3ff804f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:49,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:49,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:49,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:49,478 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679500549477"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500549477"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500549477"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500549477"}]},"ts":"1679500549477"} 2023-03-22 15:55:49,480 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-03-22 15:55:49,481 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure 9e2e5b3b5959941e4d2218f3d3ff804f, server=jenkins-hbase4.apache.org,38637,1679500416185 in 194 msec 2023-03-22 15:55:49,482 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=9e2e5b3b5959941e4d2218f3d3ff804f, ASSIGN in 353 msec 2023-03-22 15:55:49,506 INFO [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:49,508 DEBUG [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1 2023-03-22 15:55:49,508 DEBUG [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1 2023-03-22 15:55:49,508 INFO [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb17b8df8962f746a288d51c9615b0aa columnFamilyName family1 2023-03-22 15:55:49,509 INFO [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] regionserver.HStore(310): Store=eb17b8df8962f746a288d51c9615b0aa/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:49,509 INFO [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:49,510 DEBUG [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2 2023-03-22 15:55:49,510 DEBUG [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2 2023-03-22 15:55:49,511 INFO [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb17b8df8962f746a288d51c9615b0aa columnFamilyName family2 2023-03-22 15:55:49,512 INFO [StoreOpener-eb17b8df8962f746a288d51c9615b0aa-1] regionserver.HStore(310): Store=eb17b8df8962f746a288d51c9615b0aa/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:49,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:49,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:49,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:55:49,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:55:49,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:55:49,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eb17b8df8962f746a288d51c9615b0aa; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72801778, jitterRate=0.08483102917671204}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:55:49,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:55:49,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa., pid=122, masterSystemTime=1679500549436 2023-03-22 15:55:49,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:49,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:49,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:49,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 50974b1aa79ca5843b01217d351ab6aa, NAME => 'testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa.', STARTKEY => '', ENDKEY => '1'} 2023-03-22 15:55:49,519 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=eb17b8df8962f746a288d51c9615b0aa, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:49,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:49,519 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500549519"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500549519"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500549519"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500549519"}]},"ts":"1679500549519"} 2023-03-22 15:55:49,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-22 15:55:49,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:49,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:49,522 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-03-22 15:55:49,522 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure eb17b8df8962f746a288d51c9615b0aa, server=jenkins-hbase4.apache.org,38637,1679500416185 in 236 msec 2023-03-22 15:55:49,523 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=eb17b8df8962f746a288d51c9615b0aa, ASSIGN in 394 msec 2023-03-22 15:55:49,550 INFO [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:49,552 DEBUG [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1 2023-03-22 15:55:49,552 DEBUG [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1 2023-03-22 15:55:49,552 INFO [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 50974b1aa79ca5843b01217d351ab6aa columnFamilyName family1 2023-03-22 15:55:49,553 INFO [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] regionserver.HStore(310): Store=50974b1aa79ca5843b01217d351ab6aa/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:49,554 INFO [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:49,554 DEBUG [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2 2023-03-22 15:55:49,555 DEBUG [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2 2023-03-22 15:55:49,555 INFO [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 50974b1aa79ca5843b01217d351ab6aa columnFamilyName family2 2023-03-22 15:55:49,555 INFO [StoreOpener-50974b1aa79ca5843b01217d351ab6aa-1] regionserver.HStore(310): Store=50974b1aa79ca5843b01217d351ab6aa/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-22 15:55:49,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:49,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:49,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-22 15:55:49,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:55:49,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-22 15:55:49,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 50974b1aa79ca5843b01217d351ab6aa; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65088695, jitterRate=-0.030102863907814026}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-22 15:55:49,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:55:49,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa., pid=120, masterSystemTime=1679500549436 2023-03-22 15:55:49,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:49,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:49,563 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=50974b1aa79ca5843b01217d351ab6aa, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:49,563 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679500549563"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679500549563"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679500549563"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679500549563"}]},"ts":"1679500549563"} 2023-03-22 15:55:49,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-03-22 15:55:49,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure 50974b1aa79ca5843b01217d351ab6aa, server=jenkins-hbase4.apache.org,38637,1679500416185 in 280 msec 2023-03-22 15:55:49,566 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-03-22 15:55:49,566 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=50974b1aa79ca5843b01217d351ab6aa, ASSIGN in 437 msec 2023-03-22 15:55:49,567 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-22 15:55:49,567 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679500549567"}]},"ts":"1679500549567"} 2023-03-22 15:55:49,568 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-22 15:55:49,570 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-22 15:55:49,571 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.3170 sec 2023-03-22 15:55:49,972 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-22 15:55:50,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-22 15:55:50,362 INFO [Listener at localhost/43289] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-22 15:55:50,372 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:50,372 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:50,372 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:50,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:50,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-22 15:55:50,377 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-22 15:55:50,377 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:50,377 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-22 15:55:50,377 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-22 15:55:50,377 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:50,377 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:50,379 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:50,379 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:50,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:50,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:50,379 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:50,379 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:50,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:50,379 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:50,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:50,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:50,380 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-22 15:55:50,380 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-22 15:55:50,380 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-22 15:55:50,380 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:50,380 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-22 15:55:50,380 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:50,380 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. started... 2023-03-22 15:55:50,380 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:50,380 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. started... 2023-03-22 15:55:50,384 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:50,385 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9e2e5b3b5959941e4d2218f3d3ff804f 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-22 15:55:50,385 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:50,385 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:50,385 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 50974b1aa79ca5843b01217d351ab6aa 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-22 15:55:50,385 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. started... 2023-03-22 15:55:50,386 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing eb17b8df8962f746a288d51c9615b0aa 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-22 15:55:50,474 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032286eaac3a4e5c44ce81f9957e1bd67dcc to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032286eaac3a4e5c44ce81f9957e1bd67dcc 2023-03-22 15:55:50,482 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303220dd943e1a24a434ea9553888eabb89c9 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303220dd943e1a24a434ea9553888eabb89c9 2023-03-22 15:55:50,497 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/89aab35e225e42efb9ea19e01d089961 2023-03-22 15:55:50,866 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322d1585a4bbfeb43819cee00441c6d574c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230322d1585a4bbfeb43819cee00441c6d574c 2023-03-22 15:55:50,891 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/b8d346a0748546399bfbaaec99cd46f9 2023-03-22 15:55:50,926 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303223e2956e2ffd04189a02b3f37fe26786c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303223e2956e2ffd04189a02b3f37fe26786c 2023-03-22 15:55:50,931 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032268dca8ff3481460d9d0eeb6f6662a058 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023032268dca8ff3481460d9d0eeb6f6662a058 2023-03-22 15:55:51,281 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/5ae47a859f9f43b296b9adab1d5110a3 2023-03-22 15:55:51,332 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/c8cc8badff2c41389c65f168cf32b2d1 2023-03-22 15:55:51,336 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/b8d346a0748546399bfbaaec99cd46f9 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/b8d346a0748546399bfbaaec99cd46f9 2023-03-22 15:55:51,337 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/2629f371f3f948898bbce79e160f9ac3 2023-03-22 15:55:51,340 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/89aab35e225e42efb9ea19e01d089961 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/89aab35e225e42efb9ea19e01d089961 2023-03-22 15:55:51,367 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/89aab35e225e42efb9ea19e01d089961, entries=4, sequenceid=5, filesize=5.5 K 2023-03-22 15:55:51,368 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/2629f371f3f948898bbce79e160f9ac3 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/2629f371f3f948898bbce79e160f9ac3 2023-03-22 15:55:51,371 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/b8d346a0748546399bfbaaec99cd46f9, entries=4, sequenceid=5, filesize=5.5 K 2023-03-22 15:55:51,371 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/c8cc8badff2c41389c65f168cf32b2d1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/c8cc8badff2c41389c65f168cf32b2d1 2023-03-22 15:55:51,372 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/2629f371f3f948898bbce79e160f9ac3, entries=2, sequenceid=5, filesize=5.1 K 2023-03-22 15:55:51,373 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9e2e5b3b5959941e4d2218f3d3ff804f in 989ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:51,375 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:55:51,375 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:51,381 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:51,381 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/c8cc8badff2c41389c65f168cf32b2d1, entries=2, sequenceid=5, filesize=5.1 K 2023-03-22 15:55:51,382 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for eb17b8df8962f746a288d51c9615b0aa in 997ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:51,382 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:55:51,382 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:51,382 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:51,719 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032280e7ed457be649829781b46ae6c7922e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023032280e7ed457be649829781b46ae6c7922e 2023-03-22 15:55:52,125 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/86f37239c4c84a468a41bd6d082adf10 2023-03-22 15:55:52,129 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/5ae47a859f9f43b296b9adab1d5110a3 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/5ae47a859f9f43b296b9adab1d5110a3 2023-03-22 15:55:52,156 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/5ae47a859f9f43b296b9adab1d5110a3, entries=4, sequenceid=5, filesize=5.5 K 2023-03-22 15:55:52,157 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/86f37239c4c84a468a41bd6d082adf10 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/86f37239c4c84a468a41bd6d082adf10 2023-03-22 15:55:52,160 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/86f37239c4c84a468a41bd6d082adf10, entries=2, sequenceid=5, filesize=5.1 K 2023-03-22 15:55:52,161 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 50974b1aa79ca5843b01217d351ab6aa in 1777ms, sequenceid=5, compaction requested=false 2023-03-22 15:55:52,161 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:55:52,161 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:52,161 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:52,162 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:52,162 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,162 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-22 15:55:52,162 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-22 15:55:52,166 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:52,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:52,166 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,168 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,168 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:52,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:52,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:52,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:52,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-22 15:55:52,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7be80016[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:52,170 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-22 15:55:52,170 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,171 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,172 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-22 15:55:52,172 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,172 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:52,172 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-22 15:55:52,172 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:52,173 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-22 15:55:52,173 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,173 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:52,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:52,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:52,174 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-22 15:55:52,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:52,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:52,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:52,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:52,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-22 15:55:52,175 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-22 15:55:52,176 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:52,176 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:52,176 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-22 15:55:52,176 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:52,177 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,177 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:52,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:52,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,177 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:52,178 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:52,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:52,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,179 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:52,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:52,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-22 15:55:52,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:52,181 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:52,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:52,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-22 15:55:52,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:52,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:52,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:52,182 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:52,181 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:52,182 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:52,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:52,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:52,182 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,182 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,182 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,182 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,282 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:52,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:52,284 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:52,284 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:52,285 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:52,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:52,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:52,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-22 15:55:52,289 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-22 15:55:52,289 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:52,289 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-22 15:55:52,289 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-22 15:55:52,290 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,290 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:52,292 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:52,292 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:52,292 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,293 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:52,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:52,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-22 15:55:52,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-22 15:55:52,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-22 15:55:52,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-22 15:55:52,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:52,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-22 15:55:52,294 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:52,294 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. started... 2023-03-22 15:55:52,294 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:52,294 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 50974b1aa79ca5843b01217d351ab6aa 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-22 15:55:52,294 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. started... 2023-03-22 15:55:52,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:52,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:52,294 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:52,295 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. started... 2023-03-22 15:55:52,294 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9e2e5b3b5959941e4d2218f3d3ff804f 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-22 15:55:52,295 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing eb17b8df8962f746a288d51c9615b0aa 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-22 15:55:52,363 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322c3687178e2864d3a9630b70a71fa5a0f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230322c3687178e2864d3a9630b70a71fa5a0f 2023-03-22 15:55:52,374 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303224a66af5fc1c044e6886ae284f8bb8489 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303224a66af5fc1c044e6886ae284f8bb8489 2023-03-22 15:55:52,377 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032264a92b1d716e493da1f191091200b029 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032264a92b1d716e493da1f191091200b029 2023-03-22 15:55:52,387 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/8a057964c51c4da185e347297d28c7fa 2023-03-22 15:55:52,407 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/00de3bcef29c4e2ba1dbec6a24d400b0 2023-03-22 15:55:52,797 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/f49218dbfb8041e6810631373031245b 2023-03-22 15:55:52,840 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303224db788b5c264441d9fe4b7bc7415d1e0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303224db788b5c264441d9fe4b7bc7415d1e0 2023-03-22 15:55:52,849 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/8b244742ba2a490dab6a89fdb6bd8d0f 2023-03-22 15:55:52,854 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/f49218dbfb8041e6810631373031245b as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/f49218dbfb8041e6810631373031245b 2023-03-22 15:55:52,863 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303225c91156baf5c40f7bdee5ba795d35b13 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303225c91156baf5c40f7bdee5ba795d35b13 2023-03-22 15:55:52,869 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303229c1553800554477ca62cd3b5e0b03bd9 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303229c1553800554477ca62cd3b5e0b03bd9 2023-03-22 15:55:52,880 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/9c6867ab026c4817a5c9419780f1ebfa 2023-03-22 15:55:52,882 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/b52d3195048e404fa79b07f8b0c7144a 2023-03-22 15:55:52,884 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/8a057964c51c4da185e347297d28c7fa as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/8a057964c51c4da185e347297d28c7fa 2023-03-22 15:55:52,885 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/f49218dbfb8041e6810631373031245b, entries=4, sequenceid=9, filesize=5.5 K 2023-03-22 15:55:52,887 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/8b244742ba2a490dab6a89fdb6bd8d0f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/8b244742ba2a490dab6a89fdb6bd8d0f 2023-03-22 15:55:52,887 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/00de3bcef29c4e2ba1dbec6a24d400b0 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/00de3bcef29c4e2ba1dbec6a24d400b0 2023-03-22 15:55:52,892 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/8b244742ba2a490dab6a89fdb6bd8d0f, entries=2, sequenceid=9, filesize=5.1 K 2023-03-22 15:55:52,893 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 50974b1aa79ca5843b01217d351ab6aa in 599ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:52,893 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:55:52,893 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:52,893 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:52,913 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/8a057964c51c4da185e347297d28c7fa, entries=4, sequenceid=9, filesize=5.5 K 2023-03-22 15:55:52,916 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/9c6867ab026c4817a5c9419780f1ebfa as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/9c6867ab026c4817a5c9419780f1ebfa 2023-03-22 15:55:52,917 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/00de3bcef29c4e2ba1dbec6a24d400b0, entries=4, sequenceid=9, filesize=5.5 K 2023-03-22 15:55:52,918 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/b52d3195048e404fa79b07f8b0c7144a as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/b52d3195048e404fa79b07f8b0c7144a 2023-03-22 15:55:52,920 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/9c6867ab026c4817a5c9419780f1ebfa, entries=2, sequenceid=9, filesize=5.1 K 2023-03-22 15:55:52,921 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9e2e5b3b5959941e4d2218f3d3ff804f in 627ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:52,921 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:55:52,921 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:52,921 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:52,922 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/b52d3195048e404fa79b07f8b0c7144a, entries=2, sequenceid=9, filesize=5.1 K 2023-03-22 15:55:52,922 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for eb17b8df8962f746a288d51c9615b0aa in 627ms, sequenceid=9, compaction requested=false 2023-03-22 15:55:52,922 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:55:52,923 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:52,923 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:52,923 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:52,923 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,923 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-22 15:55:52,923 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-22 15:55:52,924 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:52,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:52,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:52,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:52,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,925 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,926 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,926 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:52,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:52,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-22 15:55:52,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@647cbb80[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:52,926 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-22 15:55:52,926 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,927 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,927 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-22 15:55:52,927 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-22 15:55:52,927 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:52,927 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,927 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:52,929 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-22 15:55:52,929 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:52,929 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,929 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-22 15:55:52,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:52,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:52,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:52,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:52,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:52,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:52,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-22 15:55:52,931 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-22 15:55:52,931 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:52,931 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:52,931 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-22 15:55:52,931 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:52,933 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,933 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:52,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:52,933 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:52,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,934 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:52,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:52,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:52,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:52,935 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:52,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:52,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:52,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:52,937 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:52,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:52,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-22 15:55:52,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:52,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:52,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:52,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:52,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:52,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:52,937 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:52,938 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:52,938 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:52,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:52,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-22 15:55:52,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:53,038 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:53,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:53,040 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:53,040 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:53,040 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:53,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:53,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:53,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-22 15:55:53,045 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-22 15:55:53,045 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:53,045 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-22 15:55:53,045 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-22 15:55:53,047 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:53,047 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:53,049 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:53,049 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:53,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:53,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:53,050 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:53,050 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:53,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:53,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:53,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:53,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:53,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-22 15:55:53,051 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-22 15:55:53,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-22 15:55:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-22 15:55:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-22 15:55:53,051 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:53,051 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. started... 2023-03-22 15:55:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:53,051 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:53,051 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 50974b1aa79ca5843b01217d351ab6aa 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-22 15:55:53,051 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:53,051 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. started... 2023-03-22 15:55:53,052 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. started... 2023-03-22 15:55:53,051 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:53,052 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing eb17b8df8962f746a288d51c9615b0aa 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-22 15:55:53,052 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9e2e5b3b5959941e4d2218f3d3ff804f 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-22 15:55:53,119 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322b0480b1045204f5abfcc582ea6d4387c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322b0480b1045204f5abfcc582ea6d4387c 2023-03-22 15:55:53,510 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032202964cc95bcb4ea49c693f3f93541c2c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032202964cc95bcb4ea49c693f3f93541c2c 2023-03-22 15:55:53,516 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303227175f2b2123a4e7aa9b16cb354591053 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303227175f2b2123a4e7aa9b16cb354591053 2023-03-22 15:55:53,534 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/968b7b9b17374caebc796b31c2afcf78 2023-03-22 15:55:53,544 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/6478a1a724574819862ec25a5330105e 2023-03-22 15:55:53,545 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/036c2ee3d1d54bfd934d4bd0147813ea 2023-03-22 15:55:53,623 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303221eb572dabafd4bf5a301219e46910899 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303221eb572dabafd4bf5a301219e46910899 2023-03-22 15:55:53,637 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/d0c7bd2c015342f7bf4705bb69b5d0fe 2023-03-22 15:55:53,641 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/6478a1a724574819862ec25a5330105e as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6478a1a724574819862ec25a5330105e 2023-03-22 15:55:53,675 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6478a1a724574819862ec25a5330105e, entries=4, sequenceid=13, filesize=5.5 K 2023-03-22 15:55:53,679 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/d0c7bd2c015342f7bf4705bb69b5d0fe as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/d0c7bd2c015342f7bf4705bb69b5d0fe 2023-03-22 15:55:53,684 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/d0c7bd2c015342f7bf4705bb69b5d0fe, entries=2, sequenceid=13, filesize=5.1 K 2023-03-22 15:55:53,685 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for eb17b8df8962f746a288d51c9615b0aa in 633ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:53,685 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:55:53,685 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:53,685 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:54,001 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303222fef9e5bc8f142bcb5584e9724a0ce29 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303222fef9e5bc8f142bcb5584e9724a0ce29 2023-03-22 15:55:54,016 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322b62d17692c1342ad93273c2a35b39d77 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230322b62d17692c1342ad93273c2a35b39d77 2023-03-22 15:55:54,028 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/a308d053ab48477881af69d01b1331d6 2023-03-22 15:55:54,032 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/968b7b9b17374caebc796b31c2afcf78 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/968b7b9b17374caebc796b31c2afcf78 2023-03-22 15:55:54,059 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/968b7b9b17374caebc796b31c2afcf78, entries=4, sequenceid=13, filesize=5.5 K 2023-03-22 15:55:54,060 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/a308d053ab48477881af69d01b1331d6 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/a308d053ab48477881af69d01b1331d6 2023-03-22 15:55:54,066 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/a308d053ab48477881af69d01b1331d6, entries=2, sequenceid=13, filesize=5.1 K 2023-03-22 15:55:54,067 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9e2e5b3b5959941e4d2218f3d3ff804f in 1015ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:54,067 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:55:54,067 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:54,067 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:54,434 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/e0a401c80c1646a2b342d954735a0fdf 2023-03-22 15:55:54,439 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/036c2ee3d1d54bfd934d4bd0147813ea as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/036c2ee3d1d54bfd934d4bd0147813ea 2023-03-22 15:55:54,465 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/036c2ee3d1d54bfd934d4bd0147813ea, entries=4, sequenceid=13, filesize=5.5 K 2023-03-22 15:55:54,465 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/e0a401c80c1646a2b342d954735a0fdf as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/e0a401c80c1646a2b342d954735a0fdf 2023-03-22 15:55:54,469 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/e0a401c80c1646a2b342d954735a0fdf, entries=2, sequenceid=13, filesize=5.1 K 2023-03-22 15:55:54,470 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 50974b1aa79ca5843b01217d351ab6aa in 1419ms, sequenceid=13, compaction requested=true 2023-03-22 15:55:54,470 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:55:54,470 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-22 15:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-22 15:55:54,472 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,472 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:54,472 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:54,472 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:54,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:54,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:54,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-22 15:55:54,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d556f3c[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:54,474 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-22 15:55:54,474 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:54,475 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:54,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:54,475 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-22 15:55:54,475 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-22 15:55:54,475 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:54,475 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,476 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:54,478 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,478 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-22 15:55:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,478 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:54,478 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-22 15:55:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:54,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:54,480 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-22 15:55:54,480 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-22 15:55:54,480 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:54,480 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:54,480 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-22 15:55:54,480 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:54,482 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:54,482 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:54,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:54,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:54,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:54,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:54,482 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:54,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:54,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:54,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:54,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:54,483 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:54,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:54,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:54,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:54,484 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:54,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:54,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,486 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:54,486 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:54,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:54,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-22 15:55:54,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:54,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-22 15:55:54,486 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:54,486 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:54,486 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:54,487 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:54,487 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:54,487 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:54,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:54,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:54,487 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,487 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:54,487 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:54,487 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:54,587 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:54,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:54,589 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:54,589 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:54,589 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. with WAL disabled. Data may be lost in the event of a crash. 2023-03-22 15:55:54,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:54,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:54,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-22 15:55:54,593 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-22 15:55:54,593 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:54,593 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-22 15:55:54,593 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-22 15:55:54,594 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:54,594 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:54,599 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,599 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:54,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:54,600 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:54,600 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:54,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:54,601 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-22 15:55:54,601 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-22 15:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-22 15:55:54,601 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-22 15:55:54,601 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:54,601 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-22 15:55:54,601 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:54,601 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. started... 2023-03-22 15:55:54,601 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 50974b1aa79ca5843b01217d351ab6aa 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-22 15:55:54,604 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:54,604 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. started... 2023-03-22 15:55:54,604 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9e2e5b3b5959941e4d2218f3d3ff804f 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-22 15:55:54,608 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:54,609 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:54,609 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:54,609 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. started... 2023-03-22 15:55:54,609 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing eb17b8df8962f746a288d51c9615b0aa 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-22 15:55:54,691 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303228b90de6ba52d46ef8823ac2638f454c0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303228b90de6ba52d46ef8823ac2638f454c0 2023-03-22 15:55:54,697 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303223a632c1a9ad34ea4ab262da56111f1b8 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303223a632c1a9ad34ea4ab262da56111f1b8 2023-03-22 15:55:54,717 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/68001a2d8f604b1b96ad2fa87b0e959e 2023-03-22 15:55:54,752 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303225eeeaa52dfe2451fbc940599f3a5130b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303225eeeaa52dfe2451fbc940599f3a5130b 2023-03-22 15:55:54,941 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-22 15:55:55,079 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322375c29ab66134788a95caa20b08043b7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322375c29ab66134788a95caa20b08043b7 2023-03-22 15:55:55,095 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/7cd78659dbdd4af4b382f6ecbe6b3223 2023-03-22 15:55:55,114 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/cabd4cf634aa4b3095da72f3c17903eb 2023-03-22 15:55:55,156 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/5cf57968625a4f37924d365282c976ae 2023-03-22 15:55:55,160 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/68001a2d8f604b1b96ad2fa87b0e959e as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/68001a2d8f604b1b96ad2fa87b0e959e 2023-03-22 15:55:55,167 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230322b8495ee2cceb4e49a84fa458a20a3c0a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230322b8495ee2cceb4e49a84fa458a20a3c0a 2023-03-22 15:55:55,194 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/68001a2d8f604b1b96ad2fa87b0e959e, entries=4, sequenceid=17, filesize=5.5 K 2023-03-22 15:55:55,195 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/5cf57968625a4f37924d365282c976ae as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/5cf57968625a4f37924d365282c976ae 2023-03-22 15:55:55,200 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/5cf57968625a4f37924d365282c976ae, entries=2, sequenceid=17, filesize=5.1 K 2023-03-22 15:55:55,201 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 50974b1aa79ca5843b01217d351ab6aa in 600ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:55,201 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:55:55,201 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:55,201 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:55,544 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322b88a3f747394422da44c5996a6a34330 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230322b88a3f747394422da44c5996a6a34330 2023-03-22 15:55:55,553 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/2610f7367fad4b06ad71821d4c91a040 2023-03-22 15:55:55,562 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/7cd78659dbdd4af4b382f6ecbe6b3223 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/7cd78659dbdd4af4b382f6ecbe6b3223 2023-03-22 15:55:55,583 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/382123d189184ec889ac86b45d4efced 2023-03-22 15:55:55,599 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/cabd4cf634aa4b3095da72f3c17903eb as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/cabd4cf634aa4b3095da72f3c17903eb 2023-03-22 15:55:55,600 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/7cd78659dbdd4af4b382f6ecbe6b3223, entries=4, sequenceid=17, filesize=5.5 K 2023-03-22 15:55:55,601 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/2610f7367fad4b06ad71821d4c91a040 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2610f7367fad4b06ad71821d4c91a040 2023-03-22 15:55:55,610 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2610f7367fad4b06ad71821d4c91a040, entries=2, sequenceid=17, filesize=5.1 K 2023-03-22 15:55:55,611 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for eb17b8df8962f746a288d51c9615b0aa in 1002ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:55,611 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:55:55,611 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:55,611 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:55,634 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/cabd4cf634aa4b3095da72f3c17903eb, entries=4, sequenceid=17, filesize=5.5 K 2023-03-22 15:55:55,634 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/382123d189184ec889ac86b45d4efced as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/382123d189184ec889ac86b45d4efced 2023-03-22 15:55:55,639 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/382123d189184ec889ac86b45d4efced, entries=2, sequenceid=17, filesize=5.1 K 2023-03-22 15:55:55,640 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9e2e5b3b5959941e4d2218f3d3ff804f in 1036ms, sequenceid=17, compaction requested=true 2023-03-22 15:55:55,640 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:55:55,640 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-22 15:55:55,640 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-22 15:55:55,642 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,642 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:55,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:55,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:55,642 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:55,642 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:55,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:55,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:55,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:55,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:55,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-22 15:55:55,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22ccc75e[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:55,645 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-22 15:55:55,645 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:55,646 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:55,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:55,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:55,646 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-22 15:55:55,646 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-22 15:55:55,646 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:55,646 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,646 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-22 15:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:55,649 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,649 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-22 15:55:55,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:55,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:55,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:55,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:55,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:55,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:55,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:55,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:55,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-22 15:55:55,652 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-22 15:55:55,653 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:55,653 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:55,653 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-22 15:55:55,653 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:55,654 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:55,654 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:55,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:55,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:55,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:55,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:55,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:55,654 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:55,655 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:55,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:55,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:55,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:55,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:55,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:55,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:55,656 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:55,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:55,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,658 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:55,658 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:55,658 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,658 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:55,658 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:55,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:55,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-22 15:55:55,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:55,658 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:55,658 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,658 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:55,658 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:55,658 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:55,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:55,659 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:55,659 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:55,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:55,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-22 15:55:55,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:55,759 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:55,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:55,771 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:55,778 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:55,787 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:55:55,788 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:55:55,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-22 15:55:55,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-22 15:55:55,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-22 15:55:55,911 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-22 15:55:55,911 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:55,912 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-22 15:55:55,912 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-22 15:55:55,912 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:55,912 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:55,914 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,914 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:55,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:55,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:55,914 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:55,914 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-22 15:55:55,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:55,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:55,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-22 15:55:55,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:55,915 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-22 15:55:55,916 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-22 15:55:55,916 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-22 15:55:55,916 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-22 15:55:55,916 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-22 15:55:55,916 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-22 15:55:55,928 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:55,928 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:55,928 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. started... 2023-03-22 15:55:55,928 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. started... 2023-03-22 15:55:55,928 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9e2e5b3b5959941e4d2218f3d3ff804f 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-22 15:55:55,928 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-22 15:55:55,928 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-22 15:55:55,928 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:55,928 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 50974b1aa79ca5843b01217d351ab6aa 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-22 15:55:55,929 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. started... 2023-03-22 15:55:55,929 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing eb17b8df8962f746a288d51c9615b0aa 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-22 15:55:56,402 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/838b5c6fdaee4f25a3eb636f44bf79d4 2023-03-22 15:55:56,434 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 838b5c6fdaee4f25a3eb636f44bf79d4 2023-03-22 15:55:56,437 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/0a8ada923a414c4a8a693e6c87315f76 2023-03-22 15:55:56,438 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/6c09d189642e4c49b08f12c807967ef0 2023-03-22 15:55:56,467 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c09d189642e4c49b08f12c807967ef0 2023-03-22 15:55:56,471 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a8ada923a414c4a8a693e6c87315f76 2023-03-22 15:55:56,509 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/2fd2478925a6480692afdab1635923d4 2023-03-22 15:55:56,512 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2fd2478925a6480692afdab1635923d4 2023-03-22 15:55:56,513 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/6c09d189642e4c49b08f12c807967ef0 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6c09d189642e4c49b08f12c807967ef0 2023-03-22 15:55:56,536 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/0f05864c192e49b4af2d7daa6c1f8c01 2023-03-22 15:55:56,539 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f05864c192e49b4af2d7daa6c1f8c01 2023-03-22 15:55:56,540 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/0a8ada923a414c4a8a693e6c87315f76 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/0a8ada923a414c4a8a693e6c87315f76 2023-03-22 15:55:56,553 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c09d189642e4c49b08f12c807967ef0 2023-03-22 15:55:56,553 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6c09d189642e4c49b08f12c807967ef0, entries=3, sequenceid=23, filesize=5.2 K 2023-03-22 15:55:56,559 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/2fd2478925a6480692afdab1635923d4 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2fd2478925a6480692afdab1635923d4 2023-03-22 15:55:56,567 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2fd2478925a6480692afdab1635923d4 2023-03-22 15:55:56,567 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2fd2478925a6480692afdab1635923d4, entries=1, sequenceid=23, filesize=5.0 K 2023-03-22 15:55:56,568 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for eb17b8df8962f746a288d51c9615b0aa in 639ms, sequenceid=23, compaction requested=true 2023-03-22 15:55:56,568 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:55:56,568 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:55:56,568 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-22 15:55:56,572 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a8ada923a414c4a8a693e6c87315f76 2023-03-22 15:55:56,572 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/0a8ada923a414c4a8a693e6c87315f76, entries=3, sequenceid=23, filesize=5.2 K 2023-03-22 15:55:56,573 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/0f05864c192e49b4af2d7daa6c1f8c01 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/0f05864c192e49b4af2d7daa6c1f8c01 2023-03-22 15:55:56,578 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f05864c192e49b4af2d7daa6c1f8c01 2023-03-22 15:55:56,578 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/0f05864c192e49b4af2d7daa6c1f8c01, entries=1, sequenceid=23, filesize=5.0 K 2023-03-22 15:55:56,578 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9e2e5b3b5959941e4d2218f3d3ff804f in 650ms, sequenceid=23, compaction requested=true 2023-03-22 15:55:56,579 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:55:56,579 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:56,579 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-22 15:55:57,248 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/c983a9d299fc4accb84fca8239324e2e 2023-03-22 15:55:57,252 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c983a9d299fc4accb84fca8239324e2e 2023-03-22 15:55:57,253 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/838b5c6fdaee4f25a3eb636f44bf79d4 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/838b5c6fdaee4f25a3eb636f44bf79d4 2023-03-22 15:55:57,288 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 838b5c6fdaee4f25a3eb636f44bf79d4 2023-03-22 15:55:57,288 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/838b5c6fdaee4f25a3eb636f44bf79d4, entries=3, sequenceid=23, filesize=5.2 K 2023-03-22 15:55:57,289 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/c983a9d299fc4accb84fca8239324e2e as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/c983a9d299fc4accb84fca8239324e2e 2023-03-22 15:55:57,293 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c983a9d299fc4accb84fca8239324e2e 2023-03-22 15:55:57,293 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/c983a9d299fc4accb84fca8239324e2e, entries=1, sequenceid=23, filesize=5.0 K 2023-03-22 15:55:57,293 INFO [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 50974b1aa79ca5843b01217d351ab6aa in 1365ms, sequenceid=23, compaction requested=true 2023-03-22 15:55:57,294 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:55:57,294 DEBUG [rs(jenkins-hbase4.apache.org,38637,1679500416185)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:57,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-22 15:55:57,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-22 15:55:57,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-22 15:55:57,294 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-22 15:55:57,296 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,296 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:57,296 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:57,296 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-22 15:55:57,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:57,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38637,1679500416185' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-22 15:55:57,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@191f8ba1[Count = 0] remaining members to acquire global barrier 2023-03-22 15:55:57,298 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-22 15:55:57,298 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:57,299 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:57,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:57,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:57,299 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,299 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-22 15:55:57,299 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-22 15:55:57,299 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-22 15:55:57,299 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,38637,1679500416185' in zk 2023-03-22 15:55:57,301 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:57,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:57,301 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-22 15:55:57,301 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:57,301 DEBUG [member: 'jenkins-hbase4.apache.org,38637,1679500416185' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-22 15:55:57,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:57,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:57,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:57,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:57,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:57,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,38637,1679500416185': 2023-03-22 15:55:57,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38637,1679500416185' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-22 15:55:57,304 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-22 15:55:57,304 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-22 15:55:57,304 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-22 15:55:57,304 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-22 15:55:57,304 INFO [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-22 15:55:57,310 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:57,310 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:57,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:57,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:57,310 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:57,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:57,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-22 15:55:57,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-22 15:55:57,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:57,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:57,310 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-22 15:55:57,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:57,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:57,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-22 15:55:57,311 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:57,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-22 15:55:57,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-22 15:55:57,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,315 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,315 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:57,315 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-22 15:55:57,315 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:55:57,315 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:57,315 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-22 15:55:57,315 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-22 15:55:57,315 DEBUG [(jenkins-hbase4.apache.org,37827,1679500414949)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-22 15:55:57,315 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-22 15:55:57,315 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-22 15:55:57,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-22 15:55:57,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-22 15:55:57,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-22 15:55:57,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-22 15:55:57,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-22 15:55:57,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-22 15:55:57,316 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-22 15:55:57,316 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-22 15:55:57,416 DEBUG [Listener at localhost/43289] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-22 15:55:57,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37827] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-22 15:55:57,417 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:57,417 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 50974b1aa79ca5843b01217d351ab6aa/family1 is initiating major compaction (all files) 2023-03-22 15:55:57,417 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:57,417 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:57,418 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 50974b1aa79ca5843b01217d351ab6aa/family1 in testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:57,418 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/5ae47a859f9f43b296b9adab1d5110a3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/f49218dbfb8041e6810631373031245b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/036c2ee3d1d54bfd934d4bd0147813ea, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/68001a2d8f604b1b96ad2fa87b0e959e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/838b5c6fdaee4f25a3eb636f44bf79d4] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp, totalSize=27.1 K 2023-03-22 15:55:57,418 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 5ae47a859f9f43b296b9adab1d5110a3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500550372 2023-03-22 15:55:57,419 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting f49218dbfb8041e6810631373031245b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500552284 2023-03-22 15:55:57,419 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 036c2ee3d1d54bfd934d4bd0147813ea, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500553040 2023-03-22 15:55:57,419 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 68001a2d8f604b1b96ad2fa87b0e959e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500554589 2023-03-22 15:55:57,420 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 838b5c6fdaee4f25a3eb636f44bf79d4, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:55:57,429 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 50974b1aa79ca5843b01217d351ab6aa#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:57,879 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303226e1ef9875b7a41f8bfd0c49fdaea63b4_del 2023-03-22 15:55:57,879 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303226e1ef9875b7a41f8bfd0c49fdaea63b4_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303226e1ef9875b7a41f8bfd0c49fdaea63b4_del 2023-03-22 15:55:58,748 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21de82a9dcae486d836669c3704bfa24 2023-03-22 15:55:58,749 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family1/21de82a9dcae486d836669c3704bfa24 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/21de82a9dcae486d836669c3704bfa24 2023-03-22 15:55:58,774 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21de82a9dcae486d836669c3704bfa24 2023-03-22 15:55:58,775 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 50974b1aa79ca5843b01217d351ab6aa/family1 of 50974b1aa79ca5843b01217d351ab6aa into 21de82a9dcae486d836669c3704bfa24(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:55:58,775 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:55:58,775 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:58,775 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 50974b1aa79ca5843b01217d351ab6aa/family2 is initiating major compaction (all files) 2023-03-22 15:55:58,776 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:58,776 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:58,777 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 50974b1aa79ca5843b01217d351ab6aa/family2 in testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:55:58,777 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/86f37239c4c84a468a41bd6d082adf10, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/8b244742ba2a490dab6a89fdb6bd8d0f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/e0a401c80c1646a2b342d954735a0fdf, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/5cf57968625a4f37924d365282c976ae, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/c983a9d299fc4accb84fca8239324e2e] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp, totalSize=25.4 K 2023-03-22 15:55:58,778 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 86f37239c4c84a468a41bd6d082adf10, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500550372 2023-03-22 15:55:58,779 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 8b244742ba2a490dab6a89fdb6bd8d0f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500552284 2023-03-22 15:55:58,779 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting e0a401c80c1646a2b342d954735a0fdf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500553040 2023-03-22 15:55:58,779 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 5cf57968625a4f37924d365282c976ae, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500554589 2023-03-22 15:55:58,780 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting c983a9d299fc4accb84fca8239324e2e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:55:58,797 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 50974b1aa79ca5843b01217d351ab6aa#family2#compaction#198 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:58,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/5ae47a859f9f43b296b9adab1d5110a3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/f49218dbfb8041e6810631373031245b, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/036c2ee3d1d54bfd934d4bd0147813ea, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/68001a2d8f604b1b96ad2fa87b0e959e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/838b5c6fdaee4f25a3eb636f44bf79d4] to archive 2023-03-22 15:55:58,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:58,814 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/5ae47a859f9f43b296b9adab1d5110a3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/5ae47a859f9f43b296b9adab1d5110a3 2023-03-22 15:55:58,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/f49218dbfb8041e6810631373031245b to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/f49218dbfb8041e6810631373031245b 2023-03-22 15:55:58,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/036c2ee3d1d54bfd934d4bd0147813ea to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/036c2ee3d1d54bfd934d4bd0147813ea 2023-03-22 15:55:58,819 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/68001a2d8f604b1b96ad2fa87b0e959e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/68001a2d8f604b1b96ad2fa87b0e959e 2023-03-22 15:55:58,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/838b5c6fdaee4f25a3eb636f44bf79d4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/838b5c6fdaee4f25a3eb636f44bf79d4 2023-03-22 15:55:58,829 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023032287652f9889f24aa5b3f5100b9a2dc2fc_del 2023-03-22 15:55:58,830 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032287652f9889f24aa5b3f5100b9a2dc2fc_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023032287652f9889f24aa5b3f5100b9a2dc2fc_del 2023-03-22 15:55:59,250 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1630d1b874374359a5cd4ad2a1681a65 2023-03-22 15:55:59,251 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/.tmp/family2/1630d1b874374359a5cd4ad2a1681a65 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/1630d1b874374359a5cd4ad2a1681a65 2023-03-22 15:55:59,254 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1630d1b874374359a5cd4ad2a1681a65 2023-03-22 15:55:59,255 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 50974b1aa79ca5843b01217d351ab6aa/family2 of 50974b1aa79ca5843b01217d351ab6aa into 1630d1b874374359a5cd4ad2a1681a65(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:55:59,255 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:55:59,255 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:55:59,255 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 9e2e5b3b5959941e4d2218f3d3ff804f/family1 is initiating major compaction (all files) 2023-03-22 15:55:59,256 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:55:59,256 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:55:59,256 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 9e2e5b3b5959941e4d2218f3d3ff804f/family1 in testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:55:59,256 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/89aab35e225e42efb9ea19e01d089961, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/8a057964c51c4da185e347297d28c7fa, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/968b7b9b17374caebc796b31c2afcf78, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/cabd4cf634aa4b3095da72f3c17903eb, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/0a8ada923a414c4a8a693e6c87315f76] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp, totalSize=27.1 K 2023-03-22 15:55:59,257 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 89aab35e225e42efb9ea19e01d089961, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500550372 2023-03-22 15:55:59,257 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 8a057964c51c4da185e347297d28c7fa, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500552284 2023-03-22 15:55:59,257 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 968b7b9b17374caebc796b31c2afcf78, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500553040 2023-03-22 15:55:59,258 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting cabd4cf634aa4b3095da72f3c17903eb, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500554589 2023-03-22 15:55:59,258 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 0a8ada923a414c4a8a693e6c87315f76, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:55:59,266 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 9e2e5b3b5959941e4d2218f3d3ff804f#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:55:59,309 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303225f7a40b5a81148e2a05a879baf07caee_del 2023-03-22 15:55:59,310 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303225f7a40b5a81148e2a05a879baf07caee_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303225f7a40b5a81148e2a05a879baf07caee_del 2023-03-22 15:55:59,311 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/86f37239c4c84a468a41bd6d082adf10, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/8b244742ba2a490dab6a89fdb6bd8d0f, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/e0a401c80c1646a2b342d954735a0fdf, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/5cf57968625a4f37924d365282c976ae, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/c983a9d299fc4accb84fca8239324e2e] to archive 2023-03-22 15:55:59,312 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:55:59,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/86f37239c4c84a468a41bd6d082adf10 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/86f37239c4c84a468a41bd6d082adf10 2023-03-22 15:55:59,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/8b244742ba2a490dab6a89fdb6bd8d0f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/8b244742ba2a490dab6a89fdb6bd8d0f 2023-03-22 15:55:59,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/e0a401c80c1646a2b342d954735a0fdf to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/e0a401c80c1646a2b342d954735a0fdf 2023-03-22 15:55:59,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/5cf57968625a4f37924d365282c976ae to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/5cf57968625a4f37924d365282c976ae 2023-03-22 15:55:59,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/c983a9d299fc4accb84fca8239324e2e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family2/c983a9d299fc4accb84fca8239324e2e 2023-03-22 15:56:00,173 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4309ee7b3d8a44ba9f7184e8a3157c3f 2023-03-22 15:56:00,174 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family1/4309ee7b3d8a44ba9f7184e8a3157c3f as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/4309ee7b3d8a44ba9f7184e8a3157c3f 2023-03-22 15:56:00,197 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4309ee7b3d8a44ba9f7184e8a3157c3f 2023-03-22 15:56:00,198 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9e2e5b3b5959941e4d2218f3d3ff804f/family1 of 9e2e5b3b5959941e4d2218f3d3ff804f into 4309ee7b3d8a44ba9f7184e8a3157c3f(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:56:00,198 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:56:00,198 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:56:00,198 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): 9e2e5b3b5959941e4d2218f3d3ff804f/family2 is initiating major compaction (all files) 2023-03-22 15:56:00,198 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:56:00,198 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:56:00,198 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of 9e2e5b3b5959941e4d2218f3d3ff804f/family2 in testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:56:00,199 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/2629f371f3f948898bbce79e160f9ac3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/9c6867ab026c4817a5c9419780f1ebfa, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/a308d053ab48477881af69d01b1331d6, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/382123d189184ec889ac86b45d4efced, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/0f05864c192e49b4af2d7daa6c1f8c01] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp, totalSize=25.4 K 2023-03-22 15:56:00,199 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 2629f371f3f948898bbce79e160f9ac3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500550372 2023-03-22 15:56:00,199 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 9c6867ab026c4817a5c9419780f1ebfa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500552284 2023-03-22 15:56:00,200 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting a308d053ab48477881af69d01b1331d6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500553040 2023-03-22 15:56:00,200 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 382123d189184ec889ac86b45d4efced, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500554589 2023-03-22 15:56:00,200 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 0f05864c192e49b4af2d7daa6c1f8c01, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:56:00,208 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): 9e2e5b3b5959941e4d2218f3d3ff804f#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:56:00,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/89aab35e225e42efb9ea19e01d089961, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/8a057964c51c4da185e347297d28c7fa, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/968b7b9b17374caebc796b31c2afcf78, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/cabd4cf634aa4b3095da72f3c17903eb, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/0a8ada923a414c4a8a693e6c87315f76] to archive 2023-03-22 15:56:00,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:00,212 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/89aab35e225e42efb9ea19e01d089961 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/89aab35e225e42efb9ea19e01d089961 2023-03-22 15:56:00,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/8a057964c51c4da185e347297d28c7fa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/8a057964c51c4da185e347297d28c7fa 2023-03-22 15:56:00,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/968b7b9b17374caebc796b31c2afcf78 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/968b7b9b17374caebc796b31c2afcf78 2023-03-22 15:56:00,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/cabd4cf634aa4b3095da72f3c17903eb to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/cabd4cf634aa4b3095da72f3c17903eb 2023-03-22 15:56:00,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/0a8ada923a414c4a8a693e6c87315f76 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/0a8ada923a414c4a8a693e6c87315f76 2023-03-22 15:56:00,223 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032207a8761dd88244758d9937392f1f2aaf_del 2023-03-22 15:56:00,223 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032207a8761dd88244758d9937392f1f2aaf_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023032207a8761dd88244758d9937392f1f2aaf_del 2023-03-22 15:56:01,035 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d485190fe7d4e289b03b0f2711b9600 2023-03-22 15:56:01,036 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/.tmp/family2/6d485190fe7d4e289b03b0f2711b9600 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/6d485190fe7d4e289b03b0f2711b9600 2023-03-22 15:56:01,039 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d485190fe7d4e289b03b0f2711b9600 2023-03-22 15:56:01,040 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9e2e5b3b5959941e4d2218f3d3ff804f/family2 of 9e2e5b3b5959941e4d2218f3d3ff804f into 6d485190fe7d4e289b03b0f2711b9600(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:56:01,040 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:56:01,040 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:56:01,040 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): eb17b8df8962f746a288d51c9615b0aa/family1 is initiating major compaction (all files) 2023-03-22 15:56:01,040 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:56:01,040 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:56:01,042 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of eb17b8df8962f746a288d51c9615b0aa/family1 in testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:56:01,042 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/b8d346a0748546399bfbaaec99cd46f9, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/00de3bcef29c4e2ba1dbec6a24d400b0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6478a1a724574819862ec25a5330105e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/7cd78659dbdd4af4b382f6ecbe6b3223, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6c09d189642e4c49b08f12c807967ef0] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp, totalSize=27.1 K 2023-03-22 15:56:01,042 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting b8d346a0748546399bfbaaec99cd46f9, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500550372 2023-03-22 15:56:01,042 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 00de3bcef29c4e2ba1dbec6a24d400b0, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500552285 2023-03-22 15:56:01,043 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 6478a1a724574819862ec25a5330105e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500553040 2023-03-22 15:56:01,043 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 7cd78659dbdd4af4b382f6ecbe6b3223, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500554589 2023-03-22 15:56:01,043 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 6c09d189642e4c49b08f12c807967ef0, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:56:01,049 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): eb17b8df8962f746a288d51c9615b0aa#family1#compaction#201 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:56:01,090 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230322dd482043ca5b4cd3911edebf88132ff7_del 2023-03-22 15:56:01,091 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322dd482043ca5b4cd3911edebf88132ff7_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322dd482043ca5b4cd3911edebf88132ff7_del 2023-03-22 15:56:01,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/2629f371f3f948898bbce79e160f9ac3, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/9c6867ab026c4817a5c9419780f1ebfa, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/a308d053ab48477881af69d01b1331d6, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/382123d189184ec889ac86b45d4efced, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/0f05864c192e49b4af2d7daa6c1f8c01] to archive 2023-03-22 15:56:01,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:01,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/2629f371f3f948898bbce79e160f9ac3 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/2629f371f3f948898bbce79e160f9ac3 2023-03-22 15:56:01,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/9c6867ab026c4817a5c9419780f1ebfa to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/9c6867ab026c4817a5c9419780f1ebfa 2023-03-22 15:56:01,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/a308d053ab48477881af69d01b1331d6 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/a308d053ab48477881af69d01b1331d6 2023-03-22 15:56:01,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/382123d189184ec889ac86b45d4efced to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/382123d189184ec889ac86b45d4efced 2023-03-22 15:56:01,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/0f05864c192e49b4af2d7daa6c1f8c01 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family2/0f05864c192e49b4af2d7daa6c1f8c01 2023-03-22 15:56:01,949 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:56:01,949 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x539ba0d1 to 127.0.0.1:58225 2023-03-22 15:56:01,949 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:56:01,949 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:56:01,949 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4ee95ad1 to 127.0.0.1:58225 2023-03-22 15:56:01,949 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:56:01,955 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ef7ad40ffd8a44eba5fcfcf5622864f9 2023-03-22 15:56:01,956 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family1/ef7ad40ffd8a44eba5fcfcf5622864f9 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/ef7ad40ffd8a44eba5fcfcf5622864f9 2023-03-22 15:56:01,978 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ef7ad40ffd8a44eba5fcfcf5622864f9 2023-03-22 15:56:01,979 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in eb17b8df8962f746a288d51c9615b0aa/family1 of eb17b8df8962f746a288d51c9615b0aa into ef7ad40ffd8a44eba5fcfcf5622864f9(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-22 15:56:01,979 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:56:01,979 DEBUG [Listener at localhost/43289] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-22 15:56:01,979 DEBUG [Listener at localhost/43289] regionserver.HStore(1912): eb17b8df8962f746a288d51c9615b0aa/family2 is initiating major compaction (all files) 2023-03-22 15:56:01,979 INFO [Listener at localhost/43289] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-22 15:56:01,979 INFO [Listener at localhost/43289] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-22 15:56:01,979 INFO [Listener at localhost/43289] regionserver.HRegion(2259): Starting compaction of eb17b8df8962f746a288d51c9615b0aa/family2 in testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:56:01,979 INFO [Listener at localhost/43289] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/c8cc8badff2c41389c65f168cf32b2d1, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/b52d3195048e404fa79b07f8b0c7144a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/d0c7bd2c015342f7bf4705bb69b5d0fe, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2610f7367fad4b06ad71821d4c91a040, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2fd2478925a6480692afdab1635923d4] into tmpdir=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp, totalSize=25.4 K 2023-03-22 15:56:01,980 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting c8cc8badff2c41389c65f168cf32b2d1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679500550372 2023-03-22 15:56:01,980 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting b52d3195048e404fa79b07f8b0c7144a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679500552285 2023-03-22 15:56:01,980 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting d0c7bd2c015342f7bf4705bb69b5d0fe, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679500553040 2023-03-22 15:56:01,980 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 2610f7367fad4b06ad71821d4c91a040, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679500554589 2023-03-22 15:56:01,981 DEBUG [Listener at localhost/43289] compactions.Compactor(207): Compacting 2fd2478925a6480692afdab1635923d4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-22 15:56:01,987 INFO [Listener at localhost/43289] throttle.PressureAwareThroughputController(145): eb17b8df8962f746a288d51c9615b0aa#family2#compaction#202 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-22 15:56:02,010 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/b8d346a0748546399bfbaaec99cd46f9, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/00de3bcef29c4e2ba1dbec6a24d400b0, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6478a1a724574819862ec25a5330105e, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/7cd78659dbdd4af4b382f6ecbe6b3223, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6c09d189642e4c49b08f12c807967ef0] to archive 2023-03-22 15:56:02,011 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:02,012 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/b8d346a0748546399bfbaaec99cd46f9 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/b8d346a0748546399bfbaaec99cd46f9 2023-03-22 15:56:02,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/00de3bcef29c4e2ba1dbec6a24d400b0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/00de3bcef29c4e2ba1dbec6a24d400b0 2023-03-22 15:56:02,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6478a1a724574819862ec25a5330105e to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6478a1a724574819862ec25a5330105e 2023-03-22 15:56:02,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/7cd78659dbdd4af4b382f6ecbe6b3223 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/7cd78659dbdd4af4b382f6ecbe6b3223 2023-03-22 15:56:02,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6c09d189642e4c49b08f12c807967ef0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/6c09d189642e4c49b08f12c807967ef0 2023-03-22 15:56:02,394 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230322558f5bda72294969b3417cd673225f6b_del 2023-03-22 15:56:02,394 INFO [Listener at localhost/43289] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230322558f5bda72294969b3417cd673225f6b_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230322558f5bda72294969b3417cd673225f6b_del 2023-03-22 15:56:03,206 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1d4eb0c2817840928a5faf45ab46d881 2023-03-22 15:56:03,207 DEBUG [Listener at localhost/43289] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/.tmp/family2/1d4eb0c2817840928a5faf45ab46d881 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/1d4eb0c2817840928a5faf45ab46d881 2023-03-22 15:56:03,209 INFO [Listener at localhost/43289] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1d4eb0c2817840928a5faf45ab46d881 2023-03-22 15:56:03,210 INFO [Listener at localhost/43289] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in eb17b8df8962f746a288d51c9615b0aa/family2 of eb17b8df8962f746a288d51c9615b0aa into 1d4eb0c2817840928a5faf45ab46d881(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-22 15:56:03,210 DEBUG [Listener at localhost/43289] regionserver.HRegion(2289): Compaction status journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:56:03,212 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/c8cc8badff2c41389c65f168cf32b2d1, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/b52d3195048e404fa79b07f8b0c7144a, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/d0c7bd2c015342f7bf4705bb69b5d0fe, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2610f7367fad4b06ad71821d4c91a040, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2fd2478925a6480692afdab1635923d4] to archive 2023-03-22 15:56:03,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:03,213 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:56:03,214 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/c8cc8badff2c41389c65f168cf32b2d1 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/c8cc8badff2c41389c65f168cf32b2d1 2023-03-22 15:56:03,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/b52d3195048e404fa79b07f8b0c7144a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/b52d3195048e404fa79b07f8b0c7144a 2023-03-22 15:56:03,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/d0c7bd2c015342f7bf4705bb69b5d0fe to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/d0c7bd2c015342f7bf4705bb69b5d0fe 2023-03-22 15:56:03,217 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:56:03,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2610f7367fad4b06ad71821d4c91a040 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2610f7367fad4b06ad71821d4c91a040 2023-03-22 15:56:03,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2fd2478925a6480692afdab1635923d4 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family2/2fd2478925a6480692afdab1635923d4 2023-03-22 15:56:03,220 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:56:03,221 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:56:03,225 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37827] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xA7P\x93V-'\x02\x15\xD9\xC7\xE0\xB9\xCB\x1C^\x0D"\x10\x81I\x1B\xBBw\xBD\x00{\x1BR\x8F\xFBf\xDE\xD7\xC7*\x10\xF7\x14U\x1F@"\x8F;\xA7\xBA\xF2^\x94\xCC\xA9\xDD2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-22 15:56:03,245 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-22 15:56:03,246 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-22 15:56:03,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:56:03,247 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679501163247 2023-03-22 15:56:03,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-22 15:56:03,569 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-22 15:56:03,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303226e1ef9875b7a41f8bfd0c49fdaea63b4_del 2023-03-22 15:56:03,612 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303225f7a40b5a81148e2a05a879baf07caee_del 2023-03-22 15:56:03,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230322dd482043ca5b4cd3911edebf88132ff7_del 2023-03-22 15:56:03,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-22 15:56:03,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] zookeeper.ReadOnlyZKClient(139): Connect 0x5f5ae3c0 to 127.0.0.1:58225 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-22 15:56:03,638 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c83aa9b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-22 15:56:03,639 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230322 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-22 15:56:04,590 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303222d4381614f8647c2951d768b5da465c6 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303222d4381614f8647c2951d768b5da465c6 2023-03-22 15:56:04,591 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230322 table='testMajorCompactionFromAdmin' column='family1' 2023-03-22 15:56:04,603 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-22 15:56:04,605 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56302, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-22 15:56:04,613 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-22 15:56:04,614 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40680, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-22 15:56:04,643 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230322/family1/d8ab4b04ccda44b2a4d4935dd79c7248 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-22 15:56:04,643 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230322/family1/d8ab4b04ccda44b2a4d4935dd79c7248}] 2023-03-22 15:56:04,647 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230322/family1/d8ab4b04ccda44b2a4d4935dd79c7248 for inclusion in 9e2e5b3b5959941e4d2218f3d3ff804f/family1 2023-03-22 15:56:04,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-22 15:56:04,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-22 15:56:04,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:56:04,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230322/family1/d8ab4b04ccda44b2a4d4935dd79c7248 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__ldqd8g43iu9jhgj8ugjj1bdle11082kroglpran2sbs334s7pk9p4l9s2tfjmrts/family1/d8ab4b04ccda44b2a4d4935dd79c7248 2023-03-22 15:56:04,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__ldqd8g43iu9jhgj8ugjj1bdle11082kroglpran2sbs334s7pk9p4l9s2tfjmrts/family1/d8ab4b04ccda44b2a4d4935dd79c7248 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/00ff02c83b86411d9649415cddf7a79c_SeqId_28_ 2023-03-22 15:56:04,679 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__ldqd8g43iu9jhgj8ugjj1bdle11082kroglpran2sbs334s7pk9p4l9s2tfjmrts/family1/d8ab4b04ccda44b2a4d4935dd79c7248 into 9e2e5b3b5959941e4d2218f3d3ff804f/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/00ff02c83b86411d9649415cddf7a79c_SeqId_28_ - updating store file list. 2023-03-22 15:56:04,701 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/00ff02c83b86411d9649415cddf7a79c_SeqId_28_ into 9e2e5b3b5959941e4d2218f3d3ff804f/family1 2023-03-22 15:56:04,701 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__ldqd8g43iu9jhgj8ugjj1bdle11082kroglpran2sbs334s7pk9p4l9s2tfjmrts/family1/d8ab4b04ccda44b2a4d4935dd79c7248 into 9e2e5b3b5959941e4d2218f3d3ff804f/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/family1/00ff02c83b86411d9649415cddf7a79c_SeqId_28_) 2023-03-22 15:56:04,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__ldqd8g43iu9jhgj8ugjj1bdle11082kroglpran2sbs334s7pk9p4l9s2tfjmrts/family1/d8ab4b04ccda44b2a4d4935dd79c7248 2023-03-22 15:56:04,703 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230322 table='testMajorCompactionFromAdmin' column='family1' 2023-03-22 15:56:04,704 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:04,705 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032202964cc95bcb4ea49c693f3f93541c2c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032202964cc95bcb4ea49c693f3f93541c2c 2023-03-22 15:56:04,706 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303220dd943e1a24a434ea9553888eabb89c9 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303220dd943e1a24a434ea9553888eabb89c9 2023-03-22 15:56:04,707 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303228b90de6ba52d46ef8823ac2638f454c0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303228b90de6ba52d46ef8823ac2638f454c0 2023-03-22 15:56:04,708 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230322c3687178e2864d3a9630b70a71fa5a0f to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230322c3687178e2864d3a9630b70a71fa5a0f 2023-03-22 15:56:04,708 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-22 15:56:04,708 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230322' 2023-03-22 15:56:04,708 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230322 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-22 15:56:04,708 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230322 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-22 15:56:05,248 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032288ca222d50ab4a1897e396562eb7c266 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032288ca222d50ab4a1897e396562eb7c266 2023-03-22 15:56:05,248 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='testMajorCompactionFromAdmin' column='family1' 2023-03-22 15:56:05,288 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230322/family1/6fd4c50cda3e48818f2f9616e26179ef first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-22 15:56:05,289 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230322/family1/6fd4c50cda3e48818f2f9616e26179ef}] 2023-03-22 15:56:05,290 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230322/family1/6fd4c50cda3e48818f2f9616e26179ef for inclusion in eb17b8df8962f746a288d51c9615b0aa/family1 2023-03-22 15:56:05,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-22 15:56:05,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-22 15:56:05,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:56:05,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230322/family1/6fd4c50cda3e48818f2f9616e26179ef to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__qhqbrutrtsli40gjajt1mf8qj8pepj3udjpd15ctm9of0oefggv6igpnat8vavu3/family1/6fd4c50cda3e48818f2f9616e26179ef 2023-03-22 15:56:05,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__qhqbrutrtsli40gjajt1mf8qj8pepj3udjpd15ctm9of0oefggv6igpnat8vavu3/family1/6fd4c50cda3e48818f2f9616e26179ef as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/cf1f37ecbad64c899b3c3baf79effd01_SeqId_28_ 2023-03-22 15:56:05,313 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__qhqbrutrtsli40gjajt1mf8qj8pepj3udjpd15ctm9of0oefggv6igpnat8vavu3/family1/6fd4c50cda3e48818f2f9616e26179ef into eb17b8df8962f746a288d51c9615b0aa/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/cf1f37ecbad64c899b3c3baf79effd01_SeqId_28_ - updating store file list. 2023-03-22 15:56:05,344 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/cf1f37ecbad64c899b3c3baf79effd01_SeqId_28_ into eb17b8df8962f746a288d51c9615b0aa/family1 2023-03-22 15:56:05,344 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__qhqbrutrtsli40gjajt1mf8qj8pepj3udjpd15ctm9of0oefggv6igpnat8vavu3/family1/6fd4c50cda3e48818f2f9616e26179ef into eb17b8df8962f746a288d51c9615b0aa/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/family1/cf1f37ecbad64c899b3c3baf79effd01_SeqId_28_) 2023-03-22 15:56:05,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__qhqbrutrtsli40gjajt1mf8qj8pepj3udjpd15ctm9of0oefggv6igpnat8vavu3/family1/6fd4c50cda3e48818f2f9616e26179ef 2023-03-22 15:56:05,346 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230322 table='testMajorCompactionFromAdmin' column='family1' 2023-03-22 15:56:05,346 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:05,348 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322375c29ab66134788a95caa20b08043b7 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322375c29ab66134788a95caa20b08043b7 2023-03-22 15:56:05,348 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032264a92b1d716e493da1f191091200b029 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032264a92b1d716e493da1f191091200b029 2023-03-22 15:56:05,349 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032286eaac3a4e5c44ce81f9957e1bd67dcc to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023032286eaac3a4e5c44ce81f9957e1bd67dcc 2023-03-22 15:56:05,350 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322b0480b1045204f5abfcc582ea6d4387c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322b0480b1045204f5abfcc582ea6d4387c 2023-03-22 15:56:05,350 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-22 15:56:05,350 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230322' 2023-03-22 15:56:05,350 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230322 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-22 15:56:05,350 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230322 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-22 15:56:06,289 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230322223c70fcd6764e34b7fd7428e732fe5a to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230322223c70fcd6764e34b7fd7428e732fe5a 2023-03-22 15:56:06,290 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='testMajorCompactionFromAdmin' column='family1' 2023-03-22 15:56:06,329 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230322/family1/3ff85ad468e14efe9816a855840a71f0 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-22 15:56:06,329 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa., hostname=jenkins-hbase4.apache.org,38637,1679500416185, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230322/family1/3ff85ad468e14efe9816a855840a71f0}] 2023-03-22 15:56:06,331 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(763): Validating hfile at hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230322/family1/3ff85ad468e14efe9816a855840a71f0 for inclusion in 50974b1aa79ca5843b01217d351ab6aa/family1 2023-03-22 15:56:06,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-22 15:56:06,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(777): Region bounds: first= last=1 2023-03-22 15:56:06,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegion(2446): Flush status journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:56:06,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230322/family1/3ff85ad468e14efe9816a855840a71f0 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__pls8o13g5rv6vbobmpc3j28vrq0uhfj6cpnntuigj64hiua506bt58tc6c41d4bv/family1/3ff85ad468e14efe9816a855840a71f0 2023-03-22 15:56:06,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__pls8o13g5rv6vbobmpc3j28vrq0uhfj6cpnntuigj64hiua506bt58tc6c41d4bv/family1/3ff85ad468e14efe9816a855840a71f0 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/e6f25b61e727440faf771b74af991f82_SeqId_28_ 2023-03-22 15:56:06,358 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(849): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__pls8o13g5rv6vbobmpc3j28vrq0uhfj6cpnntuigj64hiua506bt58tc6c41d4bv/family1/3ff85ad468e14efe9816a855840a71f0 into 50974b1aa79ca5843b01217d351ab6aa/family1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/e6f25b61e727440faf771b74af991f82_SeqId_28_ - updating store file list. 2023-03-22 15:56:06,378 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(882): Loaded HFile hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/e6f25b61e727440faf771b74af991f82_SeqId_28_ into 50974b1aa79ca5843b01217d351ab6aa/family1 2023-03-22 15:56:06,378 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.HStore(855): Successfully loaded hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__pls8o13g5rv6vbobmpc3j28vrq0uhfj6cpnntuigj64hiua506bt58tc6c41d4bv/family1/3ff85ad468e14efe9816a855840a71f0 into 50974b1aa79ca5843b01217d351ab6aa/family1 (new location: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/family1/e6f25b61e727440faf771b74af991f82_SeqId_28_) 2023-03-22 15:56:06,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38637] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/staging/jenkins__testMajorCompactionFromAdmin__pls8o13g5rv6vbobmpc3j28vrq0uhfj6cpnntuigj64hiua506bt58tc6c41d4bv/family1/3ff85ad468e14efe9816a855840a71f0 2023-03-22 15:56:06,380 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230322 table='testMajorCompactionFromAdmin' column='family1' 2023-03-22 15:56:06,381 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:06,382 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303223a632c1a9ad34ea4ab262da56111f1b8 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303223a632c1a9ad34ea4ab262da56111f1b8 2023-03-22 15:56:06,383 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303224a66af5fc1c044e6886ae284f8bb8489 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303224a66af5fc1c044e6886ae284f8bb8489 2023-03-22 15:56:06,384 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303227175f2b2123a4e7aa9b16cb354591053 to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303227175f2b2123a4e7aa9b16cb354591053 2023-03-22 15:56:06,384 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230322d1585a4bbfeb43819cee00441c6d574c to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230322d1585a4bbfeb43819cee00441c6d574c 2023-03-22 15:56:06,384 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-22 15:56:06,384 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230322' 2023-03-22 15:56:06,385 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230322 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-22 15:56:06,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-22 15:56:06,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-22 15:56:06,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303226e1ef9875b7a41f8bfd0c49fdaea63b4_del] 2023-03-22 15:56:06,385 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:06,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303226e1ef9875b7a41f8bfd0c49fdaea63b4_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303226e1ef9875b7a41f8bfd0c49fdaea63b4_del 2023-03-22 15:56:06,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-22 15:56:06,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303225f7a40b5a81148e2a05a879baf07caee_del] 2023-03-22 15:56:06,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:06,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303225f7a40b5a81148e2a05a879baf07caee_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303225f7a40b5a81148e2a05a879baf07caee_del 2023-03-22 15:56:06,388 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-22 15:56:06,388 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322dd482043ca5b4cd3911edebf88132ff7_del] 2023-03-22 15:56:06,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] backup.HFileArchiver(360): Archiving compacted files. 2023-03-22 15:56:06,389 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322dd482043ca5b4cd3911edebf88132ff7_del to hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230322dd482043ca5b4cd3911edebf88132ff7_del 2023-03-22 15:56:06,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-22 15:56:06,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679500420423] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-22 15:56:06,390 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-22 15:56:06,391 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.1450 sec 2023-03-22 15:56:06,404 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:56:06,406 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:56:06,408 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-22 15:56:06,409 DEBUG [Listener at localhost/43289] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-22 15:56:06,412 INFO [Listener at localhost/43289] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-22 15:56:06,447 INFO [Listener at localhost/43289] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=248 (was 266), OpenFileDescriptor=650 (was 670), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=324 (was 318) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 170), AvailableMemoryMB=6307 (was 6279) - AvailableMemoryMB LEAK? - 2023-03-22 15:56:06,447 DEBUG [Listener at localhost/43289] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b586bb5 to 127.0.0.1:58225 2023-03-22 15:56:06,448 DEBUG [Listener at localhost/43289] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:56:06,448 INFO [Listener at localhost/43289] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-22 15:56:06,448 INFO [Listener at localhost/43289] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-22 15:56:06,448 DEBUG [Listener at localhost/43289] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x561bf69b to 127.0.0.1:58225 2023-03-22 15:56:06,448 DEBUG [Listener at localhost/43289] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:56:06,448 DEBUG [Listener at localhost/43289] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-22 15:56:06,448 DEBUG [Listener at localhost/43289] util.JVMClusterUtil(257): Found active master hash=1103692770, stopped=false 2023-03-22 15:56:06,448 INFO [Listener at localhost/43289] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:56:06,450 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-22 15:56:06,450 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-22 15:56:06,450 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-22 15:56:06,450 INFO [Listener at localhost/43289] procedure2.ProcedureExecutor(629): Stopping 2023-03-22 15:56:06,451 DEBUG [Listener at localhost/43289] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64378486 to 127.0.0.1:58225 2023-03-22 15:56:06,451 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-22 15:56:06,451 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-22 15:56:06,451 DEBUG [Listener at localhost/43289] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:56:06,451 INFO [Listener at localhost/43289] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38637,1679500416185' ***** 2023-03-22 15:56:06,451 INFO [Listener at localhost/43289] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-22 15:56:06,451 INFO [RS:0;jenkins-hbase4:38637] regionserver.HeapMemoryManager(220): Stopping 2023-03-22 15:56:06,451 INFO [RS:0;jenkins-hbase4:38637] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-22 15:56:06,452 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-22 15:56:06,452 INFO [RS:0;jenkins-hbase4:38637] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-22 15:56:06,452 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 7aa87cf5a808cbf44085918270c1463d 2023-03-22 15:56:06,452 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 655cd9582db7f9e343db22d211baf616 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for c20178396b93c0ac2f980d2114beb404 2023-03-22 15:56:06,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7aa87cf5a808cbf44085918270c1463d, disabling compactions & flushes 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 9ae79c6df98f35ced4ccb3fbc1d46164 2023-03-22 15:56:06,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:56:06,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for fc08f2f22b58b2bb5e4b114f03a4e725 2023-03-22 15:56:06,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. after waiting 0 ms 2023-03-22 15:56:06,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 50974b1aa79ca5843b01217d351ab6aa 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for b3544ae53f253dc547c7d54a0e776400 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 23b8dc9b2c9ae9fdf9b83a4c4302cb43 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for d5f4b1397e7e4f07a82dabc014dab6a9 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 9e2e5b3b5959941e4d2218f3d3ff804f 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for dbbe80b35c71f0b8360fdcc3cdd8ef7e 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for eb17b8df8962f746a288d51c9615b0aa 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 9183debe8c11b7195a71e3968c944c76 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 2345ee45559d48c9feb8a4a60984e2c3 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for d05b1269482a465dff6cf06fb1c98132 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 4cb1d1539cd4a823e84ba5c732f22819 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for ec32694136837c24213a5ee256a95262 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 2c6ba5db98f0e0042380c0a57cebc9d0 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 2873f0002a080f8edd78b8fa0916e3bc 2023-03-22 15:56:06,453 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:56:06,453 DEBUG [RS:0;jenkins-hbase4:38637] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4cb57066 to 127.0.0.1:58225 2023-03-22 15:56:06,453 DEBUG [RS:0;jenkins-hbase4:38637] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:56:06,454 INFO [RS:0;jenkins-hbase4:38637] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-22 15:56:06,454 INFO [RS:0;jenkins-hbase4:38637] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-22 15:56:06,454 INFO [RS:0;jenkins-hbase4:38637] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-22 15:56:06,454 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-22 15:56:06,454 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-22 15:56:06,454 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-22 15:56:06,454 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-22 15:56:06,454 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-22 15:56:06,454 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-22 15:56:06,454 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-22 15:56:06,455 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-22 15:56:06,455 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1505): Waiting on 1588230740, 2345ee45559d48c9feb8a4a60984e2c3, 23b8dc9b2c9ae9fdf9b83a4c4302cb43, 2873f0002a080f8edd78b8fa0916e3bc, 2c6ba5db98f0e0042380c0a57cebc9d0, 4cb1d1539cd4a823e84ba5c732f22819, 50974b1aa79ca5843b01217d351ab6aa, 655cd9582db7f9e343db22d211baf616, 7aa87cf5a808cbf44085918270c1463d, 9183debe8c11b7195a71e3968c944c76, 9ae79c6df98f35ced4ccb3fbc1d46164, 9e2e5b3b5959941e4d2218f3d3ff804f, b3544ae53f253dc547c7d54a0e776400, c20178396b93c0ac2f980d2114beb404, d05b1269482a465dff6cf06fb1c98132, d5f4b1397e7e4f07a82dabc014dab6a9, dbbe80b35c71f0b8360fdcc3cdd8ef7e, eb17b8df8962f746a288d51c9615b0aa, ec32694136837c24213a5ee256a95262, fc08f2f22b58b2bb5e4b114f03a4e725, ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:56:06,457 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/7aa87cf5a808cbf44085918270c1463d/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-22 15:56:06,458 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:56:06,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7aa87cf5a808cbf44085918270c1463d: 2023-03-22 15:56:06,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679500420525.7aa87cf5a808cbf44085918270c1463d. 2023-03-22 15:56:06,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 655cd9582db7f9e343db22d211baf616, disabling compactions & flushes 2023-03-22 15:56:06,458 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:56:06,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:56:06,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. after waiting 0 ms 2023-03-22 15:56:06,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:56:06,461 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-22 15:56:06,465 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyPolicy/655cd9582db7f9e343db22d211baf616/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-22 15:56:06,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:56:06,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 655cd9582db7f9e343db22d211baf616: 2023-03-22 15:56:06,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1679500460673.655cd9582db7f9e343db22d211baf616. 2023-03-22 15:56:06,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c20178396b93c0ac2f980d2114beb404, disabling compactions & flushes 2023-03-22 15:56:06,467 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:56:06,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:56:06,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. after waiting 0 ms 2023-03-22 15:56:06,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:56:06,468 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/.tmp/info/363860c1d12042cb98a08c8a82dc0e04 2023-03-22 15:56:06,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/c20178396b93c0ac2f980d2114beb404/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-22 15:56:06,470 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:56:06,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c20178396b93c0ac2f980d2114beb404: 2023-03-22 15:56:06,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1679500456278.c20178396b93c0ac2f980d2114beb404. 2023-03-22 15:56:06,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9ae79c6df98f35ced4ccb3fbc1d46164, disabling compactions & flushes 2023-03-22 15:56:06,470 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:56:06,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:56:06,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. after waiting 0 ms 2023-03-22 15:56:06,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:56:06,471 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 363860c1d12042cb98a08c8a82dc0e04 2023-03-22 15:56:06,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/9ae79c6df98f35ced4ccb3fbc1d46164/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-22 15:56:06,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:56:06,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9ae79c6df98f35ced4ccb3fbc1d46164: 2023-03-22 15:56:06,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1679500523325.9ae79c6df98f35ced4ccb3fbc1d46164. 2023-03-22 15:56:06,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ffeec34469c62843ee9cb5963621aa5c, disabling compactions & flushes 2023-03-22 15:56:06,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:56:06,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:56:06,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. after waiting 0 ms 2023-03-22 15:56:06,474 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:56:06,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ffeec34469c62843ee9cb5963621aa5c 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-22 15:56:06,477 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/.tmp/rep_barrier/4c9a9cbe79a7409492078e2cbf9bb139 2023-03-22 15:56:06,481 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4c9a9cbe79a7409492078e2cbf9bb139 2023-03-22 15:56:06,513 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-22 15:56:06,655 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1505): Waiting on 1588230740, 2345ee45559d48c9feb8a4a60984e2c3, 23b8dc9b2c9ae9fdf9b83a4c4302cb43, 2873f0002a080f8edd78b8fa0916e3bc, 2c6ba5db98f0e0042380c0a57cebc9d0, 4cb1d1539cd4a823e84ba5c732f22819, 50974b1aa79ca5843b01217d351ab6aa, 9183debe8c11b7195a71e3968c944c76, 9e2e5b3b5959941e4d2218f3d3ff804f, b3544ae53f253dc547c7d54a0e776400, d05b1269482a465dff6cf06fb1c98132, d5f4b1397e7e4f07a82dabc014dab6a9, dbbe80b35c71f0b8360fdcc3cdd8ef7e, eb17b8df8962f746a288d51c9615b0aa, ec32694136837c24213a5ee256a95262, fc08f2f22b58b2bb5e4b114f03a4e725, ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:56:06,856 DEBUG [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1505): Waiting on 1588230740, 2345ee45559d48c9feb8a4a60984e2c3, 23b8dc9b2c9ae9fdf9b83a4c4302cb43, 2873f0002a080f8edd78b8fa0916e3bc, 2c6ba5db98f0e0042380c0a57cebc9d0, 4cb1d1539cd4a823e84ba5c732f22819, 50974b1aa79ca5843b01217d351ab6aa, 9183debe8c11b7195a71e3968c944c76, 9e2e5b3b5959941e4d2218f3d3ff804f, b3544ae53f253dc547c7d54a0e776400, d05b1269482a465dff6cf06fb1c98132, d5f4b1397e7e4f07a82dabc014dab6a9, dbbe80b35c71f0b8360fdcc3cdd8ef7e, eb17b8df8962f746a288d51c9615b0aa, ec32694136837c24213a5ee256a95262, fc08f2f22b58b2bb5e4b114f03a4e725, ffeec34469c62843ee9cb5963621aa5c 2023-03-22 15:56:06,881 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c/.tmp/info/dfebf5235691454c87a8e7039f5e718d 2023-03-22 15:56:06,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c/.tmp/info/dfebf5235691454c87a8e7039f5e718d as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c/info/dfebf5235691454c87a8e7039f5e718d 2023-03-22 15:56:06,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c/info/dfebf5235691454c87a8e7039f5e718d, entries=3, sequenceid=7, filesize=4.8 K 2023-03-22 15:56:06,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for ffeec34469c62843ee9cb5963621aa5c in 414ms, sequenceid=7, compaction requested=false 2023-03-22 15:56:06,889 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/.tmp/table/c6764adf94334dcfa28b69b60497318e 2023-03-22 15:56:06,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/namespace/ffeec34469c62843ee9cb5963621aa5c/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-22 15:56:06,892 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:56:06,892 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ffeec34469c62843ee9cb5963621aa5c: 2023-03-22 15:56:06,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1679500419297.ffeec34469c62843ee9cb5963621aa5c. 2023-03-22 15:56:06,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fc08f2f22b58b2bb5e4b114f03a4e725, disabling compactions & flushes 2023-03-22 15:56:06,893 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:56:06,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:56:06,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. after waiting 0 ms 2023-03-22 15:56:06,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:56:06,893 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c6764adf94334dcfa28b69b60497318e 2023-03-22 15:56:06,894 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/.tmp/info/363860c1d12042cb98a08c8a82dc0e04 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/info/363860c1d12042cb98a08c8a82dc0e04 2023-03-22 15:56:06,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/fc08f2f22b58b2bb5e4b114f03a4e725/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-22 15:56:06,896 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:56:06,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fc08f2f22b58b2bb5e4b114f03a4e725: 2023-03-22 15:56:06,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1679500497298.fc08f2f22b58b2bb5e4b114f03a4e725. 2023-03-22 15:56:06,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 50974b1aa79ca5843b01217d351ab6aa, disabling compactions & flushes 2023-03-22 15:56:06,897 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:56:06,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:56:06,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. after waiting 0 ms 2023-03-22 15:56:06,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:56:06,897 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 363860c1d12042cb98a08c8a82dc0e04 2023-03-22 15:56:06,897 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/info/363860c1d12042cb98a08c8a82dc0e04, entries=226, sequenceid=130, filesize=37.9 K 2023-03-22 15:56:06,898 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/.tmp/rep_barrier/4c9a9cbe79a7409492078e2cbf9bb139 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/rep_barrier/4c9a9cbe79a7409492078e2cbf9bb139 2023-03-22 15:56:06,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/50974b1aa79ca5843b01217d351ab6aa/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-22 15:56:06,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:56:06,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 50974b1aa79ca5843b01217d351ab6aa: 2023-03-22 15:56:06,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1679500548206.50974b1aa79ca5843b01217d351ab6aa. 2023-03-22 15:56:06,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b3544ae53f253dc547c7d54a0e776400, disabling compactions & flushes 2023-03-22 15:56:06,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:56:06,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:56:06,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. after waiting 0 ms 2023-03-22 15:56:06,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:56:06,902 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4c9a9cbe79a7409492078e2cbf9bb139 2023-03-22 15:56:06,902 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/rep_barrier/4c9a9cbe79a7409492078e2cbf9bb139, entries=1, sequenceid=130, filesize=5.0 K 2023-03-22 15:56:06,903 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/.tmp/table/c6764adf94334dcfa28b69b60497318e as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/table/c6764adf94334dcfa28b69b60497318e 2023-03-22 15:56:06,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/b3544ae53f253dc547c7d54a0e776400/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-22 15:56:06,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:56:06,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b3544ae53f253dc547c7d54a0e776400: 2023-03-22 15:56:06,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1679500497298.b3544ae53f253dc547c7d54a0e776400. 2023-03-22 15:56:06,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 23b8dc9b2c9ae9fdf9b83a4c4302cb43, disabling compactions & flushes 2023-03-22 15:56:06,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:56:06,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:56:06,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. after waiting 0 ms 2023-03-22 15:56:06,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:56:06,907 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c6764adf94334dcfa28b69b60497318e 2023-03-22 15:56:06,907 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/table/c6764adf94334dcfa28b69b60497318e, entries=27, sequenceid=130, filesize=6.6 K 2023-03-22 15:56:06,909 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 455ms, sequenceid=130, compaction requested=false 2023-03-22 15:56:06,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/23b8dc9b2c9ae9fdf9b83a4c4302cb43/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-22 15:56:06,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:56:06,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 23b8dc9b2c9ae9fdf9b83a4c4302cb43: 2023-03-22 15:56:06,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1679500523325.23b8dc9b2c9ae9fdf9b83a4c4302cb43. 2023-03-22 15:56:06,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d5f4b1397e7e4f07a82dabc014dab6a9, disabling compactions & flushes 2023-03-22 15:56:06,918 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:56:06,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:56:06,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. after waiting 0 ms 2023-03-22 15:56:06,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:56:06,920 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-22 15:56:06,921 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-22 15:56:06,921 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-22 15:56:06,921 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-22 15:56:06,921 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d5f4b1397e7e4f07a82dabc014dab6a9/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-22 15:56:06,921 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-22 15:56:06,922 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:56:06,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d5f4b1397e7e4f07a82dabc014dab6a9: 2023-03-22 15:56:06,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679500468144.d5f4b1397e7e4f07a82dabc014dab6a9. 2023-03-22 15:56:06,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9e2e5b3b5959941e4d2218f3d3ff804f, disabling compactions & flushes 2023-03-22 15:56:06,923 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:56:06,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:56:06,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. after waiting 0 ms 2023-03-22 15:56:06,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:56:06,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/9e2e5b3b5959941e4d2218f3d3ff804f/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-22 15:56:06,927 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:56:06,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9e2e5b3b5959941e4d2218f3d3ff804f: 2023-03-22 15:56:06,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1679500548206.9e2e5b3b5959941e4d2218f3d3ff804f. 2023-03-22 15:56:06,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dbbe80b35c71f0b8360fdcc3cdd8ef7e, disabling compactions & flushes 2023-03-22 15:56:06,927 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:56:06,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:56:06,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. after waiting 0 ms 2023-03-22 15:56:06,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:56:06,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/dbbe80b35c71f0b8360fdcc3cdd8ef7e/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-22 15:56:06,931 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:56:06,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dbbe80b35c71f0b8360fdcc3cdd8ef7e: 2023-03-22 15:56:06,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679500482021.dbbe80b35c71f0b8360fdcc3cdd8ef7e. 2023-03-22 15:56:06,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eb17b8df8962f746a288d51c9615b0aa, disabling compactions & flushes 2023-03-22 15:56:06,931 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:56:06,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:56:06,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. after waiting 0 ms 2023-03-22 15:56:06,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:56:06,939 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionFromAdmin/eb17b8df8962f746a288d51c9615b0aa/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-22 15:56:06,940 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:56:06,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eb17b8df8962f746a288d51c9615b0aa: 2023-03-22 15:56:06,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1679500548206.eb17b8df8962f746a288d51c9615b0aa. 2023-03-22 15:56:06,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9183debe8c11b7195a71e3968c944c76, disabling compactions & flushes 2023-03-22 15:56:06,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:56:06,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:56:06,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. after waiting 0 ms 2023-03-22 15:56:06,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:56:06,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/9183debe8c11b7195a71e3968c944c76/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-22 15:56:06,945 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:56:06,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9183debe8c11b7195a71e3968c944c76: 2023-03-22 15:56:06,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1679500456278.9183debe8c11b7195a71e3968c944c76. 2023-03-22 15:56:06,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2345ee45559d48c9feb8a4a60984e2c3, disabling compactions & flushes 2023-03-22 15:56:06,945 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:56:06,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:56:06,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. after waiting 0 ms 2023-03-22 15:56:06,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:56:06,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testCompactionWithHFileLink/2345ee45559d48c9feb8a4a60984e2c3/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-22 15:56:06,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:56:06,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2345ee45559d48c9feb8a4a60984e2c3: 2023-03-22 15:56:06,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1679500497298.2345ee45559d48c9feb8a4a60984e2c3. 2023-03-22 15:56:06,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d05b1269482a465dff6cf06fb1c98132, disabling compactions & flushes 2023-03-22 15:56:06,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:56:06,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:56:06,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. after waiting 0 ms 2023-03-22 15:56:06,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:56:06,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMajorCompactionWithMonthlyPolicy/d05b1269482a465dff6cf06fb1c98132/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-22 15:56:06,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:56:06,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d05b1269482a465dff6cf06fb1c98132: 2023-03-22 15:56:06,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1679500432299.d05b1269482a465dff6cf06fb1c98132. 2023-03-22 15:56:06,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4cb1d1539cd4a823e84ba5c732f22819, disabling compactions & flushes 2023-03-22 15:56:06,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:56:06,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:56:06,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. after waiting 0 ms 2023-03-22 15:56:06,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:56:06,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testScannerOnBulkLoadRefHFiles/4cb1d1539cd4a823e84ba5c732f22819/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-22 15:56:06,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:56:06,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4cb1d1539cd4a823e84ba5c732f22819: 2023-03-22 15:56:06,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1679500456278.4cb1d1539cd4a823e84ba5c732f22819. 2023-03-22 15:56:06,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ec32694136837c24213a5ee256a95262, disabling compactions & flushes 2023-03-22 15:56:06,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:56:06,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:56:06,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. after waiting 0 ms 2023-03-22 15:56:06,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:56:06,956 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithWeeklyPolicy/ec32694136837c24213a5ee256a95262/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-22 15:56:06,956 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:56:06,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ec32694136837c24213a5ee256a95262: 2023-03-22 15:56:06,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1679500445222.ec32694136837c24213a5ee256a95262. 2023-03-22 15:56:06,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2c6ba5db98f0e0042380c0a57cebc9d0, disabling compactions & flushes 2023-03-22 15:56:06,957 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:56:06,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:56:06,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. after waiting 0 ms 2023-03-22 15:56:06,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:56:06,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/ns/testMinorCompaction/2c6ba5db98f0e0042380c0a57cebc9d0/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-22 15:56:06,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:56:06,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2c6ba5db98f0e0042380c0a57cebc9d0: 2023-03-22 15:56:06,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1679500523325.2c6ba5db98f0e0042380c0a57cebc9d0. 2023-03-22 15:56:06,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2873f0002a080f8edd78b8fa0916e3bc, disabling compactions & flushes 2023-03-22 15:56:06,960 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:56:06,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:56:06,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. after waiting 0 ms 2023-03-22 15:56:06,960 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:56:06,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/data/default/testMinorCompactionWithMonthlyPolicy/2873f0002a080f8edd78b8fa0916e3bc/recovered.edits/51.seqid, newMaxSeqId=51, maxSeqId=1 2023-03-22 15:56:06,965 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:56:06,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2873f0002a080f8edd78b8fa0916e3bc: 2023-03-22 15:56:06,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1679500541718.2873f0002a080f8edd78b8fa0916e3bc. 2023-03-22 15:56:07,028 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-03-22 15:56:07,028 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-03-22 15:56:07,056 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38637,1679500416185; all regions closed. 2023-03-22 15:56:07,066 DEBUG [RS:0;jenkins-hbase4:38637] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/oldWALs 2023-03-22 15:56:07,066 INFO [RS:0;jenkins-hbase4:38637] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38637%2C1679500416185.meta:.meta(num 1679500419035) 2023-03-22 15:56:07,069 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/WALs/jenkins-hbase4.apache.org,38637,1679500416185/jenkins-hbase4.apache.org%2C38637%2C1679500416185.1679500418145 not finished, retry = 0 2023-03-22 15:56:07,172 DEBUG [RS:0;jenkins-hbase4:38637] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/oldWALs 2023-03-22 15:56:07,172 INFO [RS:0;jenkins-hbase4:38637] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38637%2C1679500416185:(num 1679500418145) 2023-03-22 15:56:07,172 DEBUG [RS:0;jenkins-hbase4:38637] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:56:07,172 INFO [RS:0;jenkins-hbase4:38637] regionserver.LeaseManager(133): Closed leases 2023-03-22 15:56:07,173 INFO [RS:0;jenkins-hbase4:38637] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-22 15:56:07,173 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-22 15:56:07,174 INFO [RS:0;jenkins-hbase4:38637] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38637 2023-03-22 15:56:07,189 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38637,1679500416185 2023-03-22 15:56:07,189 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-22 15:56:07,189 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-22 15:56:07,189 ERROR [Listener at localhost/43289-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@18fb26 rejected from java.util.concurrent.ThreadPoolExecutor@4276ffd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-22 15:56:07,189 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38637,1679500416185] 2023-03-22 15:56:07,190 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38637,1679500416185; numProcessing=1 2023-03-22 15:56:07,191 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38637,1679500416185 already deleted, retry=false 2023-03-22 15:56:07,191 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38637,1679500416185 expired; onlineServers=0 2023-03-22 15:56:07,191 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37827,1679500414949' ***** 2023-03-22 15:56:07,191 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-22 15:56:07,192 DEBUG [M:0;jenkins-hbase4:37827] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@f409d9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-22 15:56:07,192 INFO [M:0;jenkins-hbase4:37827] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37827,1679500414949 2023-03-22 15:56:07,192 INFO [M:0;jenkins-hbase4:37827] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37827,1679500414949; all regions closed. 2023-03-22 15:56:07,192 DEBUG [M:0;jenkins-hbase4:37827] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-22 15:56:07,192 DEBUG [M:0;jenkins-hbase4:37827] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-22 15:56:07,193 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-22 15:56:07,193 DEBUG [M:0;jenkins-hbase4:37827] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-22 15:56:07,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1679500417764] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1679500417764,5,FailOnTimeoutGroup] 2023-03-22 15:56:07,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1679500417764] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1679500417764,5,FailOnTimeoutGroup] 2023-03-22 15:56:07,193 INFO [M:0;jenkins-hbase4:37827] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-22 15:56:07,193 INFO [M:0;jenkins-hbase4:37827] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-22 15:56:07,193 INFO [M:0;jenkins-hbase4:37827] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-22 15:56:07,194 DEBUG [M:0;jenkins-hbase4:37827] master.HMaster(1512): Stopping service threads 2023-03-22 15:56:07,194 INFO [M:0;jenkins-hbase4:37827] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-22 15:56:07,194 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-22 15:56:07,194 ERROR [M:0;jenkins-hbase4:37827] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT 2023-03-22 15:56:07,194 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-19,5,PEWorkerGroup] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-22 15:56:07,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-22 15:56:07,195 INFO [M:0;jenkins-hbase4:37827] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-22 15:56:07,195 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-22 15:56:07,195 DEBUG [M:0;jenkins-hbase4:37827] zookeeper.ZKUtil(398): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-22 15:56:07,195 WARN [M:0;jenkins-hbase4:37827] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-22 15:56:07,195 INFO [M:0;jenkins-hbase4:37827] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-22 15:56:07,196 INFO [M:0;jenkins-hbase4:37827] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-22 15:56:07,196 DEBUG [M:0;jenkins-hbase4:37827] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-22 15:56:07,196 INFO [M:0;jenkins-hbase4:37827] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-22 15:56:07,196 DEBUG [M:0;jenkins-hbase4:37827] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-22 15:56:07,196 DEBUG [M:0;jenkins-hbase4:37827] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-22 15:56:07,196 DEBUG [M:0;jenkins-hbase4:37827] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-22 15:56:07,196 INFO [M:0;jenkins-hbase4:37827] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.70 KB heapSize=598.28 KB 2023-03-22 15:56:07,290 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-22 15:56:07,290 INFO [RS:0;jenkins-hbase4:38637] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38637,1679500416185; zookeeper connection closed. 2023-03-22 15:56:07,290 ERROR [Listener at localhost/43289-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@37287ed8 rejected from java.util.concurrent.ThreadPoolExecutor@4276ffd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-22 15:56:07,291 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@43d34866] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@43d34866 2023-03-22 15:56:07,291 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): regionserver:38637-0x1010cee76450001, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-22 15:56:07,291 ERROR [Listener at localhost/43289-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4f49a3bf rejected from java.util.concurrent.ThreadPoolExecutor@4276ffd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-22 15:56:07,291 INFO [Listener at localhost/43289] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-22 15:56:07,611 INFO [M:0;jenkins-hbase4:37827] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.70 KB at sequenceid=1003 (bloomFilter=true), to=hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7d2cbbcd485d4bd691f357cacf810dd1 2023-03-22 15:56:07,618 INFO [M:0;jenkins-hbase4:37827] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7d2cbbcd485d4bd691f357cacf810dd1 2023-03-22 15:56:07,619 DEBUG [M:0;jenkins-hbase4:37827] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7d2cbbcd485d4bd691f357cacf810dd1 as hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7d2cbbcd485d4bd691f357cacf810dd1 2023-03-22 15:56:07,624 INFO [M:0;jenkins-hbase4:37827] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7d2cbbcd485d4bd691f357cacf810dd1 2023-03-22 15:56:07,624 INFO [M:0;jenkins-hbase4:37827] regionserver.HStore(1080): Added hdfs://localhost:33875/user/jenkins/test-data/d6e9835d-4259-e684-3590-9efc5f589f98/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7d2cbbcd485d4bd691f357cacf810dd1, entries=123, sequenceid=1003, filesize=33.3 K 2023-03-22 15:56:07,625 INFO [M:0;jenkins-hbase4:37827] regionserver.HRegion(2948): Finished flush of dataSize ~512.70 KB/525001, heapSize ~598.27 KB/612624, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 429ms, sequenceid=1003, compaction requested=false 2023-03-22 15:56:07,630 INFO [M:0;jenkins-hbase4:37827] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-22 15:56:07,630 DEBUG [M:0;jenkins-hbase4:37827] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-22 15:56:07,633 INFO [M:0;jenkins-hbase4:37827] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-22 15:56:07,634 INFO [M:0;jenkins-hbase4:37827] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37827 2023-03-22 15:56:07,634 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-22 15:56:07,645 DEBUG [M:0;jenkins-hbase4:37827] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37827,1679500414949 already deleted, retry=false 2023-03-22 15:56:07,754 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-22 15:56:07,754 INFO [M:0;jenkins-hbase4:37827] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37827,1679500414949; zookeeper connection closed. 2023-03-22 15:56:07,754 ERROR [Listener at localhost/43289-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3ef4d39a rejected from java.util.concurrent.ThreadPoolExecutor@68799a52[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 905] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-22 15:56:07,754 DEBUG [Listener at localhost/43289-EventThread] zookeeper.ZKWatcher(600): master:37827-0x1010cee76450000, quorum=127.0.0.1:58225, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-22 15:56:07,754 ERROR [Listener at localhost/43289-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4f02991a rejected from java.util.concurrent.ThreadPoolExecutor@68799a52[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 905] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-22 15:56:07,756 WARN [Listener at localhost/43289] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-22 15:56:07,768 INFO [Listener at localhost/43289] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-22 15:56:07,776 WARN [BP-1976223936-172.31.14.131-1679500411633 heartbeating to localhost/127.0.0.1:33875] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-22 15:56:07,776 WARN [BP-1976223936-172.31.14.131-1679500411633 heartbeating to localhost/127.0.0.1:33875] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1976223936-172.31.14.131-1679500411633 (Datanode Uuid 4568db8c-3472-4b46-8130-137a8e5fe635) service to localhost/127.0.0.1:33875 2023-03-22 15:56:07,779 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/cluster_73db91c0-5f76-0b43-c651-cc63b18b2aa1/dfs/data/data1/current/BP-1976223936-172.31.14.131-1679500411633] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-22 15:56:07,779 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/430351c2-8e13-3549-616a-0432d67e7f36/cluster_73db91c0-5f76-0b43-c651-cc63b18b2aa1/dfs/data/data2/current/BP-1976223936-172.31.14.131-1679500411633] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-22 15:56:07,814 INFO [Listener at localhost/43289] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-22 15:56:07,958 INFO [Listener at localhost/43289] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-22 15:56:08,062 INFO [Listener at localhost/43289] hbase.HBaseTestingUtility(1293): Minicluster is down